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ence,  and  since  the  majority  of  what  it  learns  is  on-the-job,  a  wide  array 
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competition  as  leverage  to  acquire  more  effective  results.  It  offers  an  alter¬ 
native  method  for  continuous  competition— Competitive  Multisourcing 
with  Distributed  Awards— under  an  applicable  set  of  conditions  and  an 
appropriate  business  case. 
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The  federal  contracting  process  should  enable  a  government  organization 
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source  selection  processes  evaluate  how  well  a  contractor  proposes  a  solu¬ 
tion;  however,  the  government’s  processes  are  ill  suited  to  evaluate  how 
well  a  contractor  can  deliver  on  its  proposal.  The  Department  of  Defense 
(DoD)  relies  too  heavily  on  the  contractor’s  proposal  versus  evaluating  past 
performance.  The  lack  of  past  performance  data  and  processes  to  evaluate 
companies’  qualifications  has  contributed  to  program  failures,  cost  over¬ 
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mistakes  with  the  same  contractor. 


Featured  Research 


Acquisition  Program  Funding  Stability— A  Myth 
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Program  stability  and  funding  stability  are  continuously  promoted 
as  key  to  successful  acquisition  reform.  Funding  stability,  according 
to  prevailing  wisdom,  leads  to  program  stability.  Unfortunately,  the 
dynamic,  evolving,  and  methodical  requirements  generation,  tech¬ 
nology  enhancement,  and  resourcing  processes  prevalent  throughout 
the  Department  of  Defense  (DoD)  are  not  conducive  to  funding  stability. 
This  article  discusses  results  from  a  survey  of  financial  management 
practitioners  that  provide  insight  into  factors  that  both  enable  and  detract 
from  achieving  funding  stability.  The  author  presents  program  stability 
as  a  myth  in  the  real  world  environment  where  the  “norm”  is  character¬ 
ized  by  changing  program  requirements,  technologies,  and  funding.  He 
further  hypothesizes  that  stability  cannot  occur  without  major  change  in 
the  Planning,  Programming,  Budgeting  and  Execution,  and  Congressional 
Enactment  processes. 


Dynamic  Consequences  of  Cost,  Schedule,  and  Performance 
Within  DoD  Project  Management 

Patrick  R.  Cantwell,  Shahram  Sarkani,  and  Thomas  A.  Mazzuchi 

Project  management  has  been  a  constant  challenge  for  the  U.S. 
Department  of  Defense  (DoD)  acquisition  community.  While  most  DoD 
projects  are  technologically  advanced,  the  tools  and  methods  to  manage 
these  projects  are  the  same  as  for  simple,  repetitive  projects.  The  authors 
argue  that  traditional  approaches  fail  because  they  only  evaluate  the 
relationships  between  two  of  the  three  elements  of  cost,  schedule,  and 
performance.  Instead,  they  have  developed  a  system  dynamics  model 
that  allows  cost,  schedule,  and  performance  to  interact  and  influence  one 
another.  This  model  is  complementary  to  other  research  and  intended  to 
be  usable  by  the  practicing  pr oj  ect  manager.  The  results  from  model  runs 
will  provide  consequences  for  three  potential  control  alternatives  in  DoD 
project  management. 
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V  •  1 

From  the  Deputy 
Executive  Editor 

The  opening  words  of  one  of  Thomas  Paine’s 
essays  captured  the  spirit  that  guided  the  founders 
of  a  fledgling  nation  as  they  resolved  challenges  they 
faced  in  1776. 

These  are  the  times  that  try  men’s  souls.  The  summer  soldier  and 
the  sunshine  patriot  will,  in  this  crisis,  shrink  from  the  service 
of  their  country;  but  he  that  stands  by  it  now,  deserves  the  love 
of  man  and  woman.  (Paine,  1776,  p.  1) 

Ironically,  these  words  hold  true  for  contemporary  acquisition  pro¬ 
fessionals.  To  garner  the  best  possible  return  on  public  treasure  invested, 
acquisition  professionals  must  continually  strive  to  deliver  cost-effective 
systems  and  services  that  meet  users’  needs.  The  challenges  inherent  in 
achieving  that  outcome  become  more  difficult  whenever  defense  budgets 
contract,  as  they  do  on  a  cyclical  basis.  The  word  ‘crisis’  seems  appropri¬ 
ate  when  shrinking  budgets  are  squeezed  further  with  the  uncertainties 
of  continuing  resolution  and  sequestration. 

The  Defense  Acquisition  University  Alumni  Association  Hirsch 
Research  Paper  Competition  for  2013  reflected  the  idea  that  workforce 
members  must  learn  new  skills,  tools,  and  ideas  to  face  the  challenges 
they  encounter.  The  theme  for  this  year’s  competition  was:  “Improving 
the  Defense  Acquisition  Workforce  in  the  Age  of  Austerity.” 

The  first  article  in  this  issue  is  the  winning  paper  in  that  competi¬ 
tion.  Winning  authors  Col  Robert  L.  Tremaine,  USAF  (Ret.),  and  Donna 
J.  Seligman,  in  “Learning  Organizations:  Their  Importance  to  Systems 
Acquisition  in  DoD,”  conducted  research  to  highlight  how  proven  learn¬ 
ing  methods  can  be  brought  to  bear  in  government  organizations.  The 
remaining  articles  in  this  issue  illustrate  that  identifying  tried  and 
innovative  best  practices  provides  members  of  the  workforce  with  the 
tools  and  ideas  they  need  to  tackle  challenges  they  face. 

Two  of  the  articles  were  presented  as  papers  at  a  research  symposium 
hosted  at  DAU’s  Fort  Belvoir  campus  last  fall  (September  18-19,  2012) 
under  the  conference  theme  “Limits  of  Competition.”  Ginny  Wydler 
teamed  with  two  of  her  colleagues  (Su  Chang  and  Erin  M.  Schultz)  to 
explore  various  options  for  maintaining  the  benefits  of  “Continuous 


Competition  as  an  Approach  to  Maximize  Performance,”  including  mul¬ 
tisourcing  with  distributed  awards.  Also  James  Bradshaw  and  Su  Chang 
(appearing  twice  as  a  coauthor  in  this  issue),  in  “Past  Performance  as 
an  Indicator  of  Future  Performance:  Selecting  an  Industry  Partner  to 
Maximize  the  Probability  of  Program  Success,”  emphasized  the  impor¬ 
tance  of  evaluating  contractors’  past  efforts  and  key  personnel  during 
source  selection. 

COL  Robert  D.  Morig,  USA  (Ret.),  addressed  a  topic  that  is  par¬ 
ticularly  relevant  in  view  of  current  events.  His  survey  of  financial 
management  practitioners  provided  insight  into  the  factors  that  enable  or 
detract  from  achieving  funding  stability.  His  conclusions,  in  “Acquisition 
Program  Funding  Stability— A  Myth,”  might  surprise  you.  Patrick  R. 
Cantwell,  together  with  coauthors  Shahram  Sarkani  and  Thomas  A. 
Mazzuchi  from  The  George  Washington  University  faculty,  offers  a 
system  dynamic  model  in  “Dynamic  Consequences  of  Cost,  Schedule, 
and  Performance  Within  DoD  Project  Management,”  which  examines 
the  interaction  among  cost,  schedule,  and  performance  to  predict  project 
behavior  and  develop  control  strategies  for  given  situations. 

Collectively,  these  articles  reflect  the  purpose  of  acquisition 
research— providing  empirical  knowledge  that  supports  practitioners 
in  managing  and  executing  their  programs  effectively.  Particularly  in  an 
austere  age,  we  can  all  heed  words  that  also  appeared  in  Paine’s  essay: 
“The  present  winter  is  worth  an  age,  if  rightly  employed.”  As  acquisition 
practitioners,  we  must  not  “lose  or  neglect  the  opportunity”  presented 
by  austere  times  to  experiment  and  generate  knowledge  that  will  help  us 
make  more  effective  use  of  the  constrained  resources  available.  In  that 
spirit,  I  hope  you  will  enjoy  the  offerings  in  this  issue. 


< 


Dr.  Mary  C.  Redshaw 
Deputy  Executive  Editor 
Defense  ARJ 
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Research  Agenda  2013 


The  Defense  Acquisition  Research  Agenda  is  intended  to  make 
researchers  aware  of  the  topics  that  are,  or  should  be,  of  partic¬ 
ular  concern  to  the  broader  defense  acquisition  community 
throughout  the  government,  academic,  and  industrial  sectors. 
The  purpose  of  conducting  research  in  these  areas  is  to  provide 
solid,  empirically  based  findings  to  create  a  broad  body  of  knowl¬ 
edge  that  can  inform  the  development  of  policies,  procedures,  and 
processes  in  defense  acquisition,  and  to  help  shape  the  thought 
leadership  for  the  acquisition  community. 

Each  issue  of  the  Defense  AR  J  will  include  a  different  selection 
of  research  topics  from  the  overall  agenda,  which  is  at:  http:// 
www.dau.mil/research/Pages/researchareas.aspx 

Affordability  and  cost  growth 

•  Define  or  bound  “affordability”  in  the  defense  portfolio.  What  is  it? 
How  will  we  know  if  something  is  affordable  or  unaffordable? 

•  What  means  are  there  (or  can  be  developed)  to  measure,  manage,  and 
control  “affordability”  at  the  program  office  level?  At  the  industry 
level?  How  do  we  determine  their  effectiveness? 

•  What  means  are  there  (or  can  be  developed)  to  measure,  manage, 
and  control  “Should  Cost”  estimates  at  the  Service,  Component, 
program  executive,  program  office,  and  industry  levels?  How  do  we 
determine  their  effectiveness? 

•  What  means  are  there  (or  can  be  developed)  to  evaluate  and  compare 
incentives  for  achieving  “Should  Cost”  at  the  Service,  Component, 
program  executive,  program  office,  and  industry  levels? 


Recent  acquisition  studies  have  noted  the  vast  number  of  programs 
and  projects  that  do  not  make  it  successfully  through  the  acquisition 
system  and  are  subsequently  cancelled.  What  would  systematic 
root  cause  analyses  reveal  about  the  underlying  reasons,  whether 
and  how  these  cancellations  are  detrimental,  and  what  acquisition 
leaders  might  do  to  rectify  problems? 

Do  Joint  programs— at  the  inter-Service  and  international  levels— 
result  in  cost  growth  or  cost  savings  compared  with  single-Service 
(or  single-nation)  acquisition?  What  are  the  specific  mechanisms 
for  cost  savings  or  growth  at  each  stage  of  acquisition?  Do  the  data 
support  “jointness”  across  the  board,  or  only  at  specific  stages  of  a 
program,  e.g.,  only  at  research  and  development  or  only  with  specific 
aspects,  e.g.,  critical  systems  or  logistics? 

Can  we  compare  systems  with  significantly  increased  capability 
developed  in  the  commercial  market  to  DoD-developed  systems  of 
similar  characteristics? 

Is  there  a  misalignment  between  industry  and  the  government 
priorities  that  causes  the  cost  of  such  systems  to  grow  significantly 
faster  than  inflation? 

If  so,  can  we  identify  why  this  misalignment  arises?  What  relation¬ 
ship  (if  any)  does  it  have  to  industry’s  required  focus  on  shareholder 
value  and/or  profit,  versus  the  government’s  charter  to  deliver 
specific  capabilities  for  the  least  total  ownership  costs? 
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Learning  Organizations: 

Their  Importance  to  Systems 
Acquisition  in  DoD 


f  Co/  Robert  L.  Tremaine,  USAF  (Ret.), 
and  Donna  J.  Seligman 


The  success  of  the  Defense  Acquisition  Workforce 
depends  on  experience,  and  since  the  majority  of  what 
it  learns  is  on-the-job,  a  wide  array  of  learning  tech¬ 
niques  dominates.  Together,  they  behave  as  a  learning 
ecosystem  full  of  opportunities— and  even  learning 
hazards.  While  all  these  learning  techniques  jockey 
for  the  fastest  learning  lane  amid  variable  workplace 
demands,  proven  learning  methodologies  help  form 
the  foundation  of  an  organization’s  learning  faith.  Many 
organizations  already  promote  learning  in  the  workplace. 
But,  what  have  Department  of  Defense  acquisition 
organizations  that  operate  as  Learning  Organizations 
(LOs)  implemented  to  achieve  performance  gains?  The 
authors  of  this  research  sought  out  such  organizations 
to  better  understand  the  key  ingredients  that  make 
them  authentically  high-performing  and  appropriately 
armed  LOs. 


3 


Defense  ARJ,  April  2013,  Vol.  SO  No.  1 : 003-036 


Image  designed  by  Diane  Fleischer  » 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


Every  day,  organizations  face  routine  learning  challenges.  To  tackle 
them,  U.S.  organizations  spent  approximately  $156.2  billion  on  employee 
learning  in  2011  (Miller,  2012).  DoD  acquisition  organizations  that 
design,  develop,  produce,  and  maintain  essential  capabilities  required 
to  meet  U.S.  security  needs  have  instituted  their  own  learning  solu¬ 
tions.  However,  few  have  formally  adopted  all  the  learning  practices  that 
address  their  unique  learning  challenges  or  have  reenergized  previous 
learning  practices  that  have  lost  their  charge.  With  the  continued  focus 
on  finding  greater  efficiencies  in  the  workplace  coupled  with  any  com¬ 
panion  reductions  in  weapon  systems  costs,  the  concept  of  LOs  deserves 
a  closer  look  for  every  DoD  acquisition  organization  looking  to  boost 
its  learning  mileage.  Why  is  this  important?  The  DoD’s  human  capital 
workforce— acquisition  practitioners  from  all  acquisition  functional 
areas— depends  heavily  on  learning  gains,  especially  if  it  expects  to 
fulfill  warfighter  capability  needs  and  meet  Better  Buying  Power  objec¬ 
tives  promulgated  by  the  Office  of  the  Secretary  of  Defense,  which  seek 
greater  current  as  well  as  future  efficiencies  over  the  long  haul  in  weapon 
systems  procurement  for  today’s  warfighters. 

LOs  have  actually  been  around  for  some  time.  Lately,  their  relevancy 
has  come  into  question.  Some  argue  they  are  too  subjective,  elusive, 
ambiguous,  and  lack  feedback  loops  (Grieves,  2010).  Many  authors  have 
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written  about  them  or  alluded  to  them  in  some  fashion.  In  his  book  The 
Fifth  Discipline:  The  Art  and  Practice  of  The  Learning  Organization 
(1990),  Peter  Senge  first  defined  LOs  as: 

Organizations  where  people  continually  expand  their  capacity 
to  create  the  results  they  truly  desire,  where  new  and  expansive 
patterns  of  thinking  are  nurtured,  where  collective  aspiration 
is  set  free,  and  where  people  are  continually  learning  to  see  the 
whole  together,  (p.  3) 

He  further  characterized  LOs  in  the  context  of  (a)  Systems  Thinking, 
(b)  Personal  Mastery,  (c)  Mental  Models,  (d)  Shared  Visions,  and  (e) 
Team  Learning. 


Learning  vs.  Training 

Misunderstanding  the  distinction  between  formal  learning  and 
training  can  derail  the  promise  of  more  workplace  learning— a  necessity 
for  LOs.  The  difference  is  often  obscured  because  learning  and  training 
are  so  tightly  intermingled.  A  “training  experience”  is  seldom  on  the 
same  plane  as  a  “learning  experience,”  albeit  some  training  experiences, 
like  simulations,  closely  resemble  learning  experiences.  More  often  than 
not,  training  occurs  outside  the  workplace  or  work  group.  Seen  as  prepa¬ 
ratory,  training  fills  a  crucial  “know-how”  gap  where  workers  practice 
what  they  learn  without  fear  of  failure.  After  the  “training  experience”  is 
over,  workers  head  back  to  their  workplaces  and  apply  what  they  learned. 
But,  external  training  cannot  address  every  aspect  (Good  &  Brophy, 
1990).  In  the  workplace,  training  takes  the  form  of  on-the-job  training 
(OJT),  or  more  precisely  on-the-job  learning  (OJL),  and  becomes  much 
more  informal,  transparent,  ubiquitous,  and  continuous.  Mandatory 
learning  comes  back  as  formal  training  (in  the  form  of  an  intervention) 
after  something  goes  wrong  like  reduced  profits,  higher  costs,  design 
flaws,  manufacturing  defects,  safety  violations,  or  even  major  accidents 
resulting  in  loss  of  life.  Learning  in  this  context  is  not  a  continuous  activ¬ 
ity  either.  It  is  more  reactive  and  short-lived.  Understanding  how  fully 
embodied  LOs  leverage  OJL  and  other  key  learning  components  might 
help  reverse  several  other  misconceptions  about  learning  and  raise  them 
to  more  reputable  levels. 
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Reforming  Our  Thinking  About  Learning 

Despite  the  program  type  or  life-cycle  phase,  learning  in  DoD  acqui¬ 
sition  organizations  is  compulsory.  A  vast  array  of  learning  methods, 
practices,  and  techniques  prevails.  In  various  ways,  each  contributes 
to  workplace  learning.  Some  are  more  effective  than  others,  especially 
those  that  actually  mimic  the  ./oh  Far  from  a  perfect  science,  the  litera¬ 
ture  (Kerka,  1995)  suggests  effective  LOs: 

•  Provide  continuous  learning  opportunities. 

•  Use  learning  to  reach  their  goals. 

•  Link  individual  performance  with  organizational 
performance. 

•  Foster  inquiry  and  dialogue,  making  it  safe  for  people  to 
share  openly  and  take  risks. 

•  Embrace  creative  tension  as  a  source  of  energy  and  renewal. 

•  Maintain  continuous  awareness  and  interaction  with  their 
environment. 

Even  though  these  active  learning  features  help  organizations 
achieve  their  objectives,  most  organizations  have  only  a  modest  under¬ 
standing  of  how  these  features  generate  the  success  upon  which  their 
organizations  depend.  Consequently,  they  spend  less  time  thinking  about 
learning  since  future  benefits  are  not  readily  apparent.  If  DoD  organi¬ 
zations  recognized  the  significance  of  powerful  workplace  learning 
architectures,  would  they  take  them  more  seriously? 

The  researchers  selected  an  unconventional  framework  to  char¬ 
acterize  LOs  under  four  categories:  Learning  Pathways  (Lp),  Learning 
Engines  (LE),  Learning  Lubricants  (Ll),  and  Learning  Additives  (LA),  but 
used  a  traditional  mathematical  formula  to  express  them. 

n 

Learning  Organizations  =  Z  ((Lp.(LAi)  +  LEi  (LAi)  +  LLi  (LAi)) 

i=l 
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Methodology 

This  research  used  a  combination  of  interviews  and  surveys  to 
assess  learning  practices  operating  across  18  different  DoD  acquisition 
program  offices  (Figure  1).  They  constitute  a  rich  blend  of  functional 
professionals  who  apply  expertise  every  day  in  programs  spring-loaded 
with  risks  and  uncertainty.  As  a  distinctive  group,  the  researchers 
responded  that  the  current  leaders  in  DoD’s  acquisition  program  offices 
could  readily  characterize  the  learning  practices  making  a  difference  for 
them  and  the  organizations  they  lead.  Accordingly,  diverse  acquisition 
leaders  from  Acquisition  Category  (ACAT)  I  and  II  program  management 
offices,  representing  all  military  departments,  were  interviewed.  These 
DoD  acquisition  leaders  would  offer  informative  “top-down”  views.  A 
63-question  survey  was  administered  to  them  and  their  acquisition  foot 
soldiers,  who  would  offer  equally  informative  views  from  the  ground 
“looking  up.”  What  learning  attributes  made  a  difference,  and  which  ones 
required  more  learning  voltage? 


FIGURE  1.  LISTING  OF  PROGRAM  OFFICES  AND  DIRECTORATES 
INTERVIEWED  AND  SURVEYED 


18  Interviewed  and  Surveyed 


Ballistic  Missile  Defense  System  (BMDS) 

RQ-4A/B  Unmanned  Air  System  (UAS) 
GLOBAL  HAWK 

Navy  Virginia  (SSN  774)  Class  Attack 
Submarine 

Wideband  Global  Satellites  (WGS) 

(Military  Satellite  Communications  Systems 
Directorate  [MILSATCOM],  Advanced 
Extremely  High  Frequency  [AEHF],  Family 
of  Advanced  Beyond  Line-of-Sight  Terminals 
[FAB-T],  Global  Broadcast  Service  [GBS]) 

C-130  Aircraft  Modernization  Program 

National  Polar-Orbiting  Operational 
Environmental  Satellite  System  (NPOESS) 

B-2  Bomber  and  SATCOM  and  Computer 
Increment  1 

Space  Based  Infrared  System  (SBIRS)  — 

High  Satellite 

F-35  Lightning  II 

Evolved  Expendable  Launch  Vehicle  (EELV) 

C130J  -  Super  Hercules 

Global  Positioning  Systems  (GPS) 

Directorate  —  GPS  IMA  and  NAVSTAR  GPS 

MQ-9  Unmanned  Air  System  (UAS)  REAPER 

Apache  Block  IMA  (AB3A)  Remanufacture 

KC-46  Tanker 

Guided  Multiple  Launch  Rocket  System 
(GMLRS)  and  GMLRS  Alternate  Warhead 
(GMLRS-AW) 

F-22  Raptor 

Program  Executive  Office,  Command, 

Control,  Communications,  Computers  and 
Intelligence  (PEO  C4I) 
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Findings 

The  researchers  invited  4,158  acquisition  program  office  personnel 
to  take  part  in  this  survey.  Of  that  group,  2,125  personnel  responded. 
Their  aggregate  views  exposed  the  prevalence  and  dominance  of  many 
learning  components.  Their  views  also  confirmed  the  active  implemen¬ 
tation  of  16  preselected  LO  components  (independent  variables)  and  the 
resulting  workplace  learning  dividends  (dependent  variables)  expressed 
as  positive  or  negative  gaps. 


FIGURE  2.  PROGRAM  OFFICE  TOP  BOX  GAPS 

±  Aggregate  Program  Office  Top  Box 
Workplace  Learning  Gaps 


Organizational  Active  Implementation 
Individual  Learning  Dividends  Paid 


9 


Defense  ARJ,  April  3013,  Vol.  20  No.  1: 003-036 


Learning  Organizations:  Their  Importance  to  Systems  Acquisition  in  DoD 


Figure  2  represents  the  combined  percentages  for  the  top  two  boxes 
for  the  18  organizations  on  a  Likert  scale  (1-7).  Some  of  the  LO  compo¬ 
nent  percentages  were  strikingly  low.  The  subsequent  discussion 
addresses  each  component  one-by-one  by  top  box. 

Learning  Pathways  (Lp.) 

At  any  given  time,  the  direction  of  workplace  learning  matters 
(Marquardt  &  Reynolds,  1994,  p.  21).  To  give  a  clear  site  picture  of  an 
organization’s  learning  heading,  LOs  underscore  the  significance  of 
Strategic  Planning,  Organizational  Learning,  Leadership  Guidance,  and 
Learning  Climate  (Figures  3-6). 

Strategic  Planning  (Lpi).  Organizations  averaged  44  percent  for 
their  top  box  on  organizational  implementation  while  only  34  percent  in 
learning  dividends.  This  first  learning  pathway  component  emphasizes 
the  connection  to  an  organization’s  mission  and  goals.  Since  workplace 
learning  has  been  found  to  be  “the  most  effective  when  it’s  aligned  to 
corporate  objectives  and  strategies”  (The  Conference  Board  of  Canada, 
2009),  the  impacts  of  learning  outcomes  become  more  visible  when 
they  are  woven  into  an  organization’s  strategic  plan.  In  this  study, 
many  leaders  conducted  strategic  planning  initiatives.  In  their  current 
state,  the  data  indicated  conspicuously  reduced  learning  returns  for  the 
respondents.  To  increase  learning  dividends,  one  organization  made  its 
strategic  plan  a  “trusted  system”  by  instituting  a  corporate  management 
board  that  met  monthly  to  verify  worker  contributions.  The  organization 
inculcated  the  strategic  plan  into  its  learning  culture  by  tightening  the 
connection  between  individual  performance  and  mission  accomplish¬ 
ment.  In  most  organizations,  however,  strategic  plans  seemed  to  satisfy 
more  of  a  literary  requirement  than  a  means  to  a  learning  end.  Several 
leaders  considered  them  to  be  overly  burdensome  and  costly.  They 
decided  against  a  formally  written  strategic  plan  and  substituted  it  with 
“all  calls”  or  monthly/quarterly  meetings  where  they  discussed  progress 
against  their  overall  goals.  Another  organization  equated  its  Integrated 
Management  Plan  to  a  strategic  plan  since  it  anticipated  little  return  by 
investing  in  another  plan.  Over  30  years  ago,  Shell  Oil  learned  the  strong 
relationship  among  strategic  planning,  learning,  organizations,  and  cor¬ 
porate  success  (Marquardt,  2011).  DoD  acquisition  organizations  have 
not  appeared  to  find  the  same  linkage,  or  at  least  exercised  it  enough  to 
show  any  tangible  value  to  sustain  it  as  a  universal  practice.  The  work¬ 
force  was  more  confounded  by  strategic  plans.  The  respondents  who 
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rated  this  component  as  operating  below  average  responded  that  their 
plans  were  confusing,  poorly  communicated,  disconnected,  not  tracked, 
and/or  had  little  to  no  impact  on  learning. 

FIGURE  3.  STRATEGIC  PLANNING  (Lpl) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 

Organizational  Learning  (Lp2).  Organizations  averaged 36 percent 
for  their  top  box  on  organizational  implementation  and  32 percent  in  learn¬ 
ing  dividends.  Organizational  learning  forms  the  centerpiece  for  LOs  and 
incorporates  the  concept  of  adaptive  learning,  where  workers  respond  to 
changes  in  the  environment  by  detecting  errors  and  correcting  the  errors 
through  modifying  strategies,  assumptions,  or  norms  (Choo,  2006).  To 
strengthen  their  learning  bridges,  many  leaders  instituted  rotational 
assignments,  OJT  checklists,  and  hosted  recurring  “brown  bag”  discus¬ 
sions.  Others  established  microuniversities  inside  their  workplaces  that 
teach  unique  processes  and  product  line  technologies.  To  be  effective 
though,  this  second  pathway  component  requires  the  presence  of  three 
critical  factors:  meaning,  management,  and  measurement  (Garvin,  1993). 
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The  respondents  who  rated  this  component  as  operating  below  average 
reported  that  they  found  noticeable  deficits  in  all  three.  Their  organiza¬ 
tional  learning  goals  had  little  connection  to  their  work,  were  overcome 
by  program  pace,  or  lacked  meaningful  metrics. 

FIGURE  4.  ORGANIZATION  LEARNING  (Lp2) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 

Leadership  Guidance  (Lp3).  Organizations  averaged  30  percent  for 
their  top  box  on  organizational  implementation  and  34  percent  in  learn¬ 
ing  dividends.  Aside  from  serving  as  a  compass,  leaders  are  expected  to 
remove  learning  obstacles  so  their  organizations  can  make  more  learn¬ 
ing  inroads.  They  also  have  an  incumbent  responsibility  to  introduce 
workplace  “learning  initiatives. ..legitimize  managers. ..and  be  deeply 
involved  in  the  learning  process”  (Miller,  2003).  This  third  learning 
pathway  component  also  requires  leaders  to  serve  as  the  model  for 
continuous  learning  while  encouraging  their  employees  to  do  the  same. 
Often,  the  opposite  is  true  (Marquardt  &  Reynolds,  1994).  Actions  speak 
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louder  than  words.  One  leader  who  reported  higher  gains  encouraged  his 
workforce  to  seize  learning  as  their  number  one  priority  and  held  super¬ 
visors  accountable  for  making  sure  their  subordinates  gave  it  sufficient 
attention.  Several  leaders  reported  that  their  workforce  did  not  challenge 
the  status  quo  nearly  enough.  Others  expressed  the  view  that  their  daily 
demands  were  compounded  by  excessive  administrative  burden,  leaving 
them  with  less  time  to  address  all  their  learning  curves.  The  respondents 
who  rated  this  component  as  operating  below  average  said  they  needed 
much  more  definitive  direction  or  more  frequent  communication  regard¬ 
ing  learning  expectations. 

FIGURE  5.  LEADERSHIP  GUIDANCE  (Lp3) 


Implementation  x=  30% 


Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 

Learning  Climate  (Lp4).  Organizations  averaged  37  percent  for  their 
top  box  on  organizational  implementation  and  39  percent  in  learning 
dividends.  This  last  pathway  component  speaks  to  the  workplace  safe¬ 
guards  in  place  to  mitigate  the  learning  turbulence  that  can  emanate 
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from  leadership  expectations,  workplace  processes,  or  workplace  cul¬ 
tures.  Effective  LOs  ground  these  key  elements  by  instituting  resilient 
and  sustainable  learning  practices  that  encourage  and  condition  their 
employees  to  value  the  need  to  continually  learn  new  skills  and  “avoid 
the  erosion  of  their  knowledge  stocks”  (Cooke  &  Meyer,  2007).  One  leader 
offered  that  he  actively  pushes  his  workforce  to  think  critically  and  chal¬ 
lenge  the  status  quo.  He  further  reported  that  his  organization  could 
never  meet  its  technical  challenges  without  it.  Another  leader  reminded 
his  workforce  to  actively  think  differently.  Respondents  who  worked  in 
organizations  where  this  component  rated  below  average  reported  their 
learning  climates  were  too  weak  to  face  the  pressures  of  risk.  People  took 
shelter  to  avoid  it  since  their  leadership  did  not  endorse  it. 

FIGURE  6.  LEARNING  CLIMATE  (Lp4) 
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Learning  Engine  (LEj) 

Learning  engines  are  the  source  of  an  organization’s  learning  muscle. 
They  depend  heavily  on  individual  learning,  increased  responsibility, 
professional  development,  and  individual  advancement  (Figures  7-10). 
Properly  sized  learning  engines  give  organizations  the  ability  to  tackle 
uncertain  and  variable  learning  terrain  with  lesser  strain.  Learning 
engines  also  have  to  operate  at  peak  levels  to  achieve  enough  momentum 
to  safely  negotiate  steep  learning  grades. 

Individual  Learning  (LE1).  Organizations  averaged  33  percent  for 
their  top  box  on  organizational  implementation  and  41  percent  in  learn¬ 
ing  dividends.  A  vehicle’s  towing  capacity  depends  on  the  horsepower 
and  torque  its  engine  produces.  In  a  similar  way,  individual  learning 
represents  the  source  of  an  organization’s  intellectual  muscle.  Like  any 
muscle,  it  needs  to  be  exercised.  Individuals  must  value  and  keep  their 
new  learning  skills  fit  enough  to  promote  “psychological  states  of  com¬ 
petence”  (Cooke  &  Meyer,  2007).  This  first  learning  engine  component 
is  closely  linked  with  Lp2  in  an  explicit  and  structured  way  (Marquardt 
&  Reynolds,  1994).  Individual  learning  gives  organizations  immediate 
traction  by  serving  as  a  “core  resource  and  mechanism”  that  moves  orga¬ 
nizations  toward  their  goals  (Srihawong,  Srisa-Ard,  &  Chiwpimai,  2012). 
It  also  helps  organizations  respond  to  strong  learning  counterforces  like 
competition  from  other  workplace  demands  and  daily  programmatic 
risks  that  subject  individuals  to  continuous  learning  pressure.  To  help 
strengthen  individual  learning  development,  one  leader  had  his  junior 
personnel  teach  others  what  they  had  learned.  He  ensured  they  had 
learning  in  the  correct  gear  so  they  could  effectively  react  to  workplace 
eventualities  while  operating  at  peak  proficiency  levels.  The  respondents 
who  reported  below  average  dividends  questioned  the  amount  of  time  set 
aside  for  individual  learning,  or  the  link  between  learning  and  perfor¬ 
mance  improvements  was  missing. 
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FIGURE  7.  INDIVIDUAL  LEARNING  (LE1) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 


Increased  Responsibility  (LE2).  Organizations  averaged  43  percent 
for  their  top  box  on  organizational  implementation  and  42 percent  in  learn¬ 
ing  dividends.  LOs  are  known  to  evenly  distribute  responsibility  across 
their  enterprises  in  the  same  fashion  that  air  shocks  and  assisted  breaking 
systems  safely  handle  heavier  loads  “on  demand.”  Although  occasionally 
tenuous,  this  second  learning  engine  component  also  keeps  employees 
intellectually  challenged  enough  so  they  do  not  seek  employment  else¬ 
where  (Emery,  2010).  One  leader  reminded  his  workforce  that  “Innovation 
doesn’t  live  in  the  routine,  and  takes  persistence  and  the  responsibility  to 
challenge  themselves  instead.”  The  opposing  forces  (e.g.,  lack  of  motivation 
and  shortage  of  available  time)  can  inhibit  the  pursuit  for  some  workers  to 
seek  or  accept  increasingly  more  responsibility.  However,  the  distribution 
of  responsibility  deserves  frequent  inspection  since  it  behaves  as  a  catalyst 
for  forces  leading  to  change  management  inside  LOs  (Beaver  &  Hutchings, 
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2004).  The  respondents  who  reported  lower  than  average  results  cited  the 
preponderance  of  responsibility  placed  on  select  positions  as  not  always 
evenly  distributed,  minimized,  or  even  overlooked. 

FIGURE  8.  INCREASED  RESPONSIBILITY  (LE2) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 

Professional  Development  (LE3).  Organizations  averaged  36 
percent  for  their  top  box  on  organizational  implementation  and  40 
percent  in  learning  dividends.  Professional  development  helps  learn¬ 
ing  engines  burn  leaner  by  improving  learners’  “time  to  competence” 
(Senge,  1990).  Additional  knowledge  found  in  collaborative  opportunities 
like  professional  conferences,  communities  of  practice,  or  cooperative 
deep  intellectual  dives  on  functionally  specific  topics  favorably  boost 
learning  effects.  Internal  development  programs  make  strong  impacts 
since  they  are  more  workplace-specific.  One  leader  crafted  an  internal 
Career  Development  Guide  that  created  a  comprehensive  glide  path  for 
a  wide  range  of  experiential  and  collaborative  learning  opportunities 
inside  his  learning  house.  Another  leader  modified  his  organization’s 
reporting  structure  to  allow  more  junior  personnel  to  assume  roles  that 
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increased  their  developmental  momentum.  The  respondents  who  rated 
this  third  learning  engine  component  as  below  average  reported  that 
professional  development  was  either  poorly  promoted,  unorganized,  ad 
hoc,  or  inactive. 

FIGURE  9.  PROFESSIONAL  DEVELOPMENT  (LEJ) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 


Individual  Advancement  (LE4).  Organizations  averaged  35 per¬ 
cent  for  their  top  box  on  organizational  implementation  and  39  percent 
in  learning  dividends.  LOs  help  their  workforce  seek  advancement  by 
applying  more  force  to  their  learning  opportunity  accelerator.  One  leader 
whose  organization  reported  the  highest  workforce  learning  dividends  in 
this  last  learning  engine  component  instituted  (a)  functionally  focused 
internal  meetings  to  show  what  it  takes  for  personnel  to  advance;  (b)  a 
program  where  competitive  individuals  could  diversify  into  other  func¬ 
tional  areas;  and  (c)  an  accession  model  that  illustrated  the  experience 
required  for  progression.  Interestingly  enough,  advancement  does  not 
always  imply  more  supervision,  which  could  be  holding  back  some  from 
seeking  it  (Kosteas,  2011).  Respondents  who  reported  below  average 
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advancement  opportunities  expressed  the  view  that  more  promising 
prospects  existed  outside  their  own  workplaces  or  lacked  the  time  to 
pursue  the  required  qualifications  to  compete  for  internal  advancement. 

FIGURE  10.  INDIVIDUAL  ADVANCEMENT  (LE4) 


Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 


Learning  Lubricants  (Lu) 

Purposeful,  timely,  and  active  learning  in  the  workplace  is  an 
important  component  for  organizational  success.  But,  under  this  third 
category,  friction  can  easily  interfere  with  expected  gains  if  four  com¬ 
ponents— empowerment,  mentorship,  individual  feedback,  and  creative 
tension— are  not  at  their  ideal  viscous  states.  The  variable  and  unrelent¬ 
ing  learning  pace  found  inside  acquisition  organizations  requires  all  four 
components  to  keep  workplace  learning  moving  freely  (Figures  11-14). 
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Empowerment  (Ll1).  Organizations  averaged  38  percent  for  their 
top  box  on  organizational  implementation  and  43  percent  in  learning 
dividends.  When  it  comes  to  learning,  empowerment  might  be  the  most 
highly  underestimated  component  of  them  all.  In  this  study,  it  signaled 
the  highest  individual  learning  dividends  paid.  Companies  like  General 
Electric  actively  push  empowerment  by  applying  a  risk  quotient  where 
they  “measure  employee  performance  based  on  their  capacity  to  take 
risk  in  championing  ideas,  learn  from  the  experience,  and  drive  improve¬ 
ment”  (Peters,  2012).  Leaders  who  reported  high  learning  dividends 
from  empowerment  widely  delegated  “the  authority”  across  their  orga¬ 
nizations.  Respondents  in  organizations  that  operated  below  average 
reported  that  empowerment  was  visibly  absent,  not  fostered,  or  under¬ 
whelming  at  their  workplaces. 

FIGURE  11.  EMPOWERMENT  (l_L1) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 
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Mentorship  (LL2).  Organizations  averaged  27  percent  for  their 
top  box  on  organizational  implementation  and  35  percent  in  learning 
dividends.  LOs  seize  mentorship  since  it  helps  employees  avoid  costly 
mistakes.  LOs  also  recognize  that  mentors  must  be  willing  to  bear  the 
responsibility  for  their  employees’  growth  and  development  in  their 
dual  role  as  a  “performance  confronter”  and  “career  counselor”  (Gilley 
&  Maycunich,  2000,  p.  32).  One  leader  noted  that  making  mentorship 
too  formal  would  lead  to  its  death.  He  selected  certain  personnel  to  fill 
positions  that  demanded  mentorship.  The  respondents  who  reported 
below  average  dividends  for  this  second  lubricant  component  saw  little 
evidence  of  mentorship  even  though  they  felt  it  could  pay  huge  returns 
if  it  found  its  way  into  their  development. 

FIGURE  12.  MENTORSHIP  (l_L2) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 
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Individual  Feedback  (LL3).  Organizations  averaged  26  percent  for 
their  top  box  on  organizational  implementation  and  35 percent  in  learning 
dividends.  LOs  recognize  the  importance  of  feedback— the  only  facet  of 
knowledge  and  skill  development  that  is  significantly  associated  with 
individual  impact  (Cooke  &  Meyer,  2007).  In  its  raw  form,  this  third 
learning  lubricant  operates  like  a  learning  performance  regulator.  Too 
little  feedback  can  slow  the  learning  flow.  Too  much  feedback  can  lead 
to  excessive  focus  where  learners  are  always  altering  their  performance, 
leading  to  inconsistent  and  variable  performance-impaired  learning 
(Lee  &  Carnahan,  1990).  Premature  feedback  can  have  an  adverse  learn¬ 
ing  effect  much  like  an  engine  backfires  when  an  explosion  occurs  in  the 
air  intake  or  exhaust  system  rather  than  inside  the  combustion  chamber. 
Negative  feedback  can  be  toxic  and  contaminate  learning  climates.  In 
its  ideal  form,  feedback  needs  to  be  timely,  respectful,  accurate,  care¬ 
fully  communicated,  and  void  of  negative  undertones.  Leadership  plays 
a  significant  role  in  feedback  by  ensuring  it  remains  constructive  and 
freely  flows,  but  sticky  enough  to  reduce  workplace  propaganda  and 
eliminate  counterproductive  interference.  Most  leaders  reported  that 
feedback  directly  affects  their  ability  to  accomplish  workplace  chal¬ 
lenges  and  made  it  a  priority  across  their  organizations.  The  respondents 
who  experienced  below  average  learning  dividends  noted  either  little  or 
less  constructive  feedback,  no  connection  to  learning  plans,  or  a  failure 
to  close  the  feedback  loop. 
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FIGURE  13.  INDIVIDUAL  FEEDBACK  (LL3) 
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Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 

Creative  Tension  (LL4).  Organizations  averaged  31  percent  for  their 
top  box  on  organizational  implementation  and  37 percent  in  learning 
dividends.  LOs  encourage  their  personnel  to  seek  new  learning  methods 
and  embrace  creative  tension  as  a  positive  attribute  because  it  gener¬ 
ates  resolution  (Senge,  1990).  One  leader  stitched  healthy  tension  into 
his  own  learning  formula  and  encouraged  his  workforce  to  voice  their 
disagreement  at  every  meeting  if  they  felt  strongly  about  an  issue.  He 
could  not  think  of  a  better  way  for  them  to  shoulder  more  “ownership”  at 
the  workplace.  Some  respondents  misunderstood  the  concept  of  this  last 
learning  lubricant,  but  the  respondents  who  noted  lower  than  average 
dividends  reported  little  evidence  of  tension  in  their  workplace,  espe¬ 
cially  the  creative  type,  and  it  resulted  in  missed  learning  opportunities. 
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FIGURE  14.  CREATIVE  TENSION  (l_L4) 


70% 


60% 

m  50% 

ii 

lx 

10 

•a 

g  40% 

T3 

■> 

5 

30% 


20% 

20%  30%  40% 

Implementation  x  =  31% 


Note.  Each  dot  represents  aggregate  top  box  responses  from  the  18  program  offices. 


• 

• 

• 

•  •  • 
• 

Defense ARJ,  April 2013,  Vol.  20 No.  1: 003-036 


24 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


Learning  Additives  (LAj) 

LOs  recognize  the  need  for  certain  learning  additives  under  this  last 
category  such  as  new  technologies,  challenging  work,  time  for  learning, 
and  generational  learning  solutions  (Figures  15  -18).  They  give  workplace 
learning  added  momentum  and  can  raise  learning  outcomes  to  even  more 
favorable  levels.  This  last  category  evaluated  the  effectiveness  of  each. 

Learning  Technology  (LA1).  Organizations  averaged  46 percent 
for  their  top  box  that  technology  was  effectively  used.  Technologies  are 
becoming  more  and  more  predominant  in  the  workplace.  They  can 
help  organizations  save  money,  save  time,  increase  productivity,  man¬ 
age  knowledge,  and  improve  learning.  In  the  last  several  years,  social 
media  has  skyrocketed.  In  an  earlier  survey  that  polled  125  learning 
and  training  leaders,  82  percent  used  social  media  to  advance  their  own 
professional  skills  and  resources  while  another  81  percent  believed 
social  media  offers  valuable  learning  opportunities  (The  CARA  Group, 
2010).  In  another  study,  Twitter®  and  YouTube®  ranked  number  one  and 
two,  respectively,  as  tools  for  workplace  learning  among  545  learning 
professionals  worldwide  (Hart,  2011).  In  this  LO  study,  e-mail  was  seen 
as  the  most  effective  learning  technology,  although  it  also  created  issues 
(Figure  6).  Several  program  managers  instituted  more  restrictive  e-mail 
discipline  to  reduce  the  e-mail  barrage  by  instituting  no  more  “reply  to 
all”  and  no  more  e-mails  to  their  leadership  team  without  “action  rec¬ 
ommendations.”  Another  reminded  his  personnel  to  “send  less  so  they 
would  get  less.”  One  in  particular  issued  an  e-mail  “stand-down”  day 
and  directed  his  personnel  to  either  communicate  by  phone  or  talk  face- 
to-face.  Afterwards,  he  noticed  a  shift  in  cooperative  learning.  People 
started  to  talk  again  and  shared  knowledge  more  openly.  The  low  rating 
of  social  media  in  acquisition  organizations  could  most  likely  be  attrib¬ 
uted  to  limited  access  to  certain  sites.  Generational  preferences  may  also 
play  a  role  since  far  fewer  “millennials”  are  yet  working  in  acquisition 
organizations.  Nonetheless,  learning  technologies  serve  as  agateway  to 
both  information  and  knowledge  sharing.  However,  some  organizations 
in  this  study  had  limited  means  to  leverage  more  effective  solutions  or 
the  knowledge  to  understand  this  first  additive’s  association  to  learning. 
Many  key  learning  technology  decisions  were  left  to  the  information 
technology  specialists. 
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FIGURE  15.  LEARNING  TECHNOLOGY  (LA1) 


EMAIL 


VTCS,  ONLINE  MEETINGS 

ONLINE  LEARNING 

MOBILE  DEVICES 

INTRANET/KNOWLEDGE  P 

ORTAL 

KNOWLEDGE  DATABASES 
INSTANT  MESSAGING 
SHAREPOINT 


0%  10%  20%  30%  40%  50%  60%  70% 

Effectiveness 


Challenging  Work  (LA2).  Organizations  averaged  65 percent  top 
box  for  presenting  challenging  work.  Adding  challenging  work  into  the 
learning  mix  helps  individuals  achieve  greater  self-efficiency  (Huys,  De 
Rick,  &  Vandenbrande,  2005).  One  leader  said  that  until  he  got  his  people 
exposed  to  this  second  learning  additive,  he  risked  losing  them.  Another 
leader  encouraged  his  personnel  to  read  the  book  StrengthsFinder  2.0  by 
Tom  Rath,  and  then  had  them  list  five  strengths  to  share  with  others.  He 
reported  that  the  organization  as  a  whole  could  achieve  more  challenging 
work  if  it  understood  the  sum  of  its  parts. 
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FIGURE  16.  CHALLENGING  WORK  (LA2) 
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Time  for  Learning  (LA3).  Organizations  averaged  41  percent  top 
box  for  giving  enough  time  to  master  skill.  For  workplace  learning  to  be 
meaningful,  LOs  allow  adequate  time  for  learning  to  “warm  up”  and  give 
learners  time  to  reflect,  practice,  network,  and  seek  any  necessary  train¬ 
ing  (Vaughan,  2008).  Many  leaders  blended  “just-in-time”  learning  into 
their  learning  mixtures  whenever  new  processes  or  initiatives  surfaced. 
Others  reinforced  the  importance  of  taking  time  to  build  expertise. 
One  leader  reminded  his  personnel  not  to  leave  the  organization  with¬ 
out  becoming  proficient  in  their  fields.  Another  leader  created  time  for 
thinking  experiments  inside  his  organization.  One  of  his  teams  decided 
to  run  a  product  line  contest  out  of  graham  crackers,  peanut  butter  and 
marshmallows,  and  toothpicks.  To  them,  the  competition  ended  up  rein¬ 
forcing  the  importance  of  product  resiliency  and  a  resilient  workforce. 
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FIGURE  17.  TIME  FOR  LEARNING  (LA3) 
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Generational  Learning  Accommodations  (LA4).  Organizations 
averaged  26  percent  top  box  for  accommodating  differences  in  genera¬ 
tional  learning.  Looming  changes  in  workforce  demographics  have  placed 
even  more  pressure  on  an  organization’s  learning  ecosystem.  However, 
while  generations  have  their  own  learning  preferences,  how  they  actu¬ 
ally  learn  is  not  significant  enough  to  “warrant  different  instructional 
designs  or  learning  technologies”  (Reeves,  2006).  None  of  the  leaders 
instituted  any  generational-unique  learning  techniques  although  many 
leaders  reported  that  they  gave  more  attention  to  the  development  of  their 
junior  workforce.  One  leader  ensured  his  junior  personnel  understood 
that  performance  would  evolve  them  as  “hot  runners.”  Another  leader 
specified  that  teaching  the  next  generation  at  his  workplace  was  the  most 
important  thing  he  could  do. 
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FIGURE  18.  GENERATIONAL  LEARNING  (LA4) 


Recommendations 

With  a  conspicuous  mix  of  entry  to  senior-level  personnel  who  run 
the  experience  scale,  the  acquisition  workforce  demonstrates  a  wide 
range  of  “know  how”  that  constantly  fluctuates.  While  they  relish  what 
they  learn  on  the  job,  few  fully  appreciate  the  magnitude  of  all  the  learn¬ 
ing  elements  that  affect  their  learning  development.  Even  though  the 
DoD  organizations  surveyed  in  this  study  confirmed  the  presence  of 
all  the  LO  architectural  components,  no  single  acquisition  organiza¬ 
tion  has  fully  energized  them  all.  Based  on  extrapolation,  more  active 
implementation  could  result  in  a  stronger  learning  footing  and  create 
more  positive  learning  dividends  for  every  individual  and  organization. 
Consequently,  the  researchers  recommend  the  following  for  those  in  a 
position  to  champion  the  learning  charge: 
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Become  your  organization’s  Chief  Learning  Officer.  Take  the 
time  to  understand  all  the  key  learning  practices  that  should  be  preva¬ 
lent  and  highly  active  in  your  organization.  Assess  their  contribution  to 
mission  outcomes.  Involve  yourself  in  your  organization’s  total  learning 
equation.  If  you  haven’t  yet  done  so: 

•  Energize  your  strategic  plan.  Communicate  it  and  measure 
progress  against  it.  Whatever  the  manifestation,  it  needs 
to  be  grounded,  connected  to  both  individual  and  orga¬ 
nizational  outcomes,  flexible,  well-communicated,  and 
understood. 

•  Codify  your  organization’s  OJL  program.  It  is  where  most 
workplace  learning  occurs,  and  organizational  competence 
depends  on  it  (Olmstead,  2002).  Decide  what  needs  to  be 
formal  and  what  does  not. 

•  Recognize  that  learning  and  formal  training  are  distinctively 
different.  Remind  your  workforce  that  learning  is  more 
formal  and  incidental.  Learning  is  a  contact  sport.  Make 
time  to  reflect. 

•  Monitor  your  learning  climates  closely.  Inspire  and  condi¬ 
tion  your  workforce  to  value  the  need  to  continually  learn 
new  skills  to  avoid  the  erosion  of  its  knowledge  stocks. 
Promulgate  the  virtues  of  innovative  thinking. 

•  Eliminate  the  seam  between  “time  for  doing”  and  “time  for 
learning.”  The  difference  is  too  close  to  call.  "Doing”  is 
experiential  learning. 

•  Distribute  responsibility  across  your  enterprise.  It  increases 
learning  health  and  reduces  personnel  turnover. 

•  Create  opportunities  for  professional  development.  It  pro¬ 
duces  greater  depths  of  expertise  and  strengthens  an 
organization’s  learning  core. 

•  Encourage  advancement.  It  makes  workers  think  more 
about  their  own  skillsets  and  how  they  can  make  even 
greater  impacts. 
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•  Empower  your  people  and  give  them  a  solid  sense  of  respon¬ 
sibility.  It  increases  their  learning  capacity  and  reinforces 
their  confidence.  Give  your  personnel  permission  to  switch 
gears.  Encourage  them  to  take  risks. 

•  Make  mentorship  a  top  priority  and  actively  promote  it. 
Mentors  help  build  more  sustainable  careers  for  junior 
workers  who  are  running  low  on  experience. 

•  Provide  more  performance  feedback.  There  is  no  stronger 
learning  barometer. 

•  Embrace  creative  tension.  Ask  your  workforce  where 
your  organization  needs  to  be  (i.e.,  vision)  versus  the  “as 
is.”  Explain  that  any  gap  between  the  two  restricts  the 
achievement  of  critical  outcomes.  Allow  your  workforce  to 
challenge  the  status  quo  in  a  thoughtful  and  respectful  way. 

•  Maintain  learning  agility.  Whenever  learning  needs  change, 
maintain  agility  (e.g.,  presence  of  interns,  changes  in  mis¬ 
sion,  changes  in  personnel,  etc.). 

•  Strategically  manage  your  technology  needs.  Ground  them  to 
organizational  goals.  Don’t  let  them  short-circuit  the  ability 
to  get  work  done  (Allen,  2012). 

Conclusions 

People  have  always  been  an  organization’s  secret  weapon,  and  no 
cutting-edge  system  capability  could  have  ever  been  built  let  alone  con¬ 
ceived  without  it.  After  22  years  since  their  inception,  LOs  are  still  very 
relevant  since  learning  is  omnipresent  in  the  workplace.  It  may  be  hard  to 
visualize,  but  fully  embodied  LOs  can  help  DoD  acquisition  practitioners 
think  more  deliberately  about  effective  learning  solutions.  Indeed,  LOs 
can  provide  just  enough  escape  velocity  to  leave  less  productive  learn¬ 
ing  practices  behind,  including  the  patterns  that  could  be  undermining 
learning  itself,  and  ultimately— over  the  long  haul— help  raise  learning 
to  more  efficient  levels. 
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Continuous  Competition 
as  an  Approach  to 
Maximize  Performance 

i  Ginny  Wydler,  Su  Chang,  and  Erin  M.  Schultz 


Research  shows  that  continuing  competitive  pressure 
applied  during  development  and  production  leads  to 
better  industry  performance,  often  at  reduced  cost. 
However,  the  entrenched  practice  of  one-time  competi¬ 
tion  for  an  entire  program  life  cycle  often  endows  the 
winner  with  a  very  strong  monopolistic  power  that 
lasts  for  decades.  This  article  describes  continuous 
competition  as  leverage  to  acquire  more  effective 
results.  It  offers  an  alternative  method  for  continuous 
competition— Competitive  Multisourcing  with  Distributed 
Awards— under  an  applicable  set  of  conditions  and  an 
appropriate  business  case. 
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Competition  has  always  been  a  cornerstone  of  the  Department  of  the 
Defense  (DoD)’s  acquisition  system.  Competitive  contract  awards  have 
yielded  notable  benefits,  including  reduced  weapon  systems  costs  and 
improved  contractor  performance,  efficiencies,  and  innovation.  However, 
competition  rarely  continues  after  contract  award.  Instead,  the  winning 
contractor  often  establishes  monopolistic  advantages  and  gains  “vendor 
lock.”  As  a  result,  programs  can  experience  cost  growth,  schedule  delays, 
and  average  or  even  poor  performance.  By  contrast,  experience  has 
shown  that  continuous  competition  can  drive  both  incremental  improve¬ 
ment  and  game-changing  innovation  in  weapon  systems  acquisition. 

This  article  proposes  that  the  DoD  adopt  an  innovative  form  of 
continuous  competition:  Competitive  Multisourcing  with  Distributed 
Awards.  Under  this  approach,  DoD  programs  would  select  more  than 
one  contractor  to  develop,  produce,  and  sustain  a  program  throughout 
its  life  cycle:  a  primary  source  and  one  or  more  secondary  sources  that 
contribute  a  lower  level  of  design  development  and  work  share. 

As  envisioned,  this  approach  offers  multiple  benefits  to  the  DoD. 

•  Having  multiple  contractors  would  provide  programs  with 
an  “insurance  policy”  in  the  event  that  the  primary  contrac¬ 
tor  fails  to  perform  adequately.  Thus,  it  would  reduce  single 
points  of  failure— a  critical  advantage  in  today’s  increas¬ 
ingly  complex  and  interdependent  acquisition  environment. 

•  The  existence  of  one  or  more  alternative  providers  can 
spur  the  prime  contractor  to  maintain  a  high  level  of  per¬ 
formance  by  maintaining  the  pressure  of  price  competition 
throughout  the  system  life  cycle.  It  would  also  help  prevent 
monopolistic  behavior  by  the  primary  contractor. 

•  The  risks  inherent  in  contract  transition  would  be  mini¬ 
mized  by  the  availability  of  a  second  source  that  is  already 
knowledgeable  about  the  day-to-day  operations  of  the 
program.  Should  the  prime  contractor  fail  to  meet  DoD 
expectations,  the  second  source  could  quickly  ramp  up  to 
fill  the  primary  contractor’s  obligations. 
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•  The  expanded  opportunity  to  win  a  share  of  a  large  contract 

should  stimulate  additional  companies  to  develop  their 
capabilities  and  become  credible  competitors.  This  would 
address  widespread  concern  about  the  shrinking  defense 
industrial  base,  in  which  four  companies  have  dominated 
the  market  since  1999. 

Competitive  Multisourcing  with  Distributed  Awards  would  have 
particular  benefits  for  certain  types  of  programs,  including  those  that 
have  applied  dual  sourcing  successfully  in  the  past.  It  would  also  enable 
the  DoD  to  take  advantage  of  two  product  designs  rather  than  choosing 
a  single  solution  and  foregoing  the  features  of  a  promising  alternative. 
Engaging  a  low-level  second  source  early  in  the  program  could  generate 
high  pay-off  in  production.  Finally,  programs  with  high  cost  overruns 
could  apply  Competitive  Multisourcing  with  Distributed  Awards  to 
engage  a  low-level  second  source  in  developing  additional  technology. 
This  would  encourage  both  contractors  to  refine  and  mature  their  techni¬ 
cal  designs,  thus  permitting  better  cost  analysis  and  cost  containment. 
This  article  describes  the  current  defense  contracting  environment, 
and  shows  how  the  practice  of  awarding  single  contracts  has  led  to 
unnecessarily  high  costs  and  poor  performance.  It  then  uses  historical 
examples  to  illustrate  the  benefits  of  a  dual-source  approach,  and  pres¬ 
ents  examples  of  continuous  competition.  Finally,  the  article  presents 
Competitive  Multisourcing  with  Distributed  Awards  as  a  recommended 
form  of  continuing  competition,  and  examines  the  advantages  that  such 
an  approach  would  bring  to  the  DoD. 

Competition  in  Defense  Contracting 

Both  government  and  industry  have  long  recognized  the  benefits  of 
competition  in  improving  performance  and  reducing  costs.  The  value  of 
competition  has  been  incorporated  into  every  major  piece  of  legislation 
on  acquisition  reform  and  is  continually  touted  in  political  speeches  and 
public  engagements.  However,  the  vast  majority  of  DoD  programs  con¬ 
tinue  to  rely  on  a  single-provider  acquisition  approach  and  spend  most 
of  their  life  cycles  without  real  and  enduring  competition.  As  a  result, 
too  many  DoD  acquisition  programs  fail  to  achieve  their  cost,  schedule, 
and  performance  objectives. 
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The  absence  of  a  continuous  competitive  force,  such  as  a  credible 
industry  competitor,  may  contribute  to  this  situation.  Once  a  contrac¬ 
tor  is  chosen  to  develop  a  major  new  system,  the  DoD  is  locked  into  a 
relationship  with  that  contractor  that  could  last  20  years  or  more  (Arena 
&  Birkler,  2009,  p.  5).  As  a  result  of  this  de  facto  monopoly,  the  DoD  has 
little  opportunity  to  obtain  broad  insight  regarding  engineering  changes 
or  technology  upgrades,  other  than  from  the  prime  contractor,  and  that 
contractor  may  not  be  entirely  objective. 

Congress  enacted  the  Competition  in  Contracting  Act  of  1984 
and  later  reaffirmed  its  importance  by  passing  the  Weapon  Systems 
Acquisition  Reform  Act  (WSARA)  of  2009.  Under  this  new  legislation. 
Congress  mandated  that  the  Secretary  of  Defense  require  that  each 
major  defense  acquisition  program  (MDAP)  include  in  its  acquisition 
strategies  measures  to  ensure  competition,  or  the  option  of  competition, 
throughout  a  program’s  life  cycle  (Weapon  Systems,  2009,  §  202,  p.  18). 
Dr.  Ashton  Carter,  then  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics,  emphasized  competitive  contracting  as  one  of 
the  five  “affordability  tools”  to  be  utilized  within  the  DoD  in  his  “Better 
Buying  Power”  memorandum  of  2010  (Carter,  2010,  p.  9).  With  recent 
declining  budgets,  the  competitive  environment  is  at  risk  because  the 
DoD  lacks  the  funds  to  invest  in  making  companies  capable  of  challeng¬ 
ing  the  top  defense  contractors. 

In  spite  of  the  attention  focused  on  improving  competition,  major 
systems  acquisition  continues  to  experience  problems  in  develop¬ 
ment  and  production.  Fewer  than  half  of  the  programs  in  DoD’s  2010 
MDAP  portfolio  meet  the  established  performance  metrics  and  cost- 
performance  goals  agreed  to  by  the  DoD,  the  Office  of  Management  and 
Budget,  and  the  Government  Accountability  Office  (GAO)  (GAO,  2011a, 
p.  6).  According  to  a  study  by  the  Center  for  Strategic  and  International 
Studies  (CSIS),  32  percent  of  the  programs  awarded  competitively  expe¬ 
rienced  cost  overruns,  amounting  to  $19  billion  (Hofbauer,  Sanders, 
Ellman,  &  Morrow,  2011,  p.  11).  A  recent  GAO  study  on  “Trends  in  Nunn- 
McCurdy  Cost  Breaches  for  Major  Defense  Acquisition  Programs” 
identified  74  breaches  involving  47  major  programs  from  1997  to  2011 
(GAO,  2011b,  p.  5).  Breaches  occur  when  unit  costs  increase  by  at  least 
25  percent  over  the  current  baseline  estimate  or  at  least  50  percent  over 
the  original  baseline  estimate.  GAO’s  analysis  showed  that  nearly  half  of 
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Nunn-McCurdy  breaches  occurred  after  a  production  decision  had  been 
made— when  a  program  has  fewer  options  for  restructuring.  For  example, 
the  DDG-1000  Destroyer  and  F-35  Joint  Strike  Fighter  (JSF)  were 
required  to  recertify  and  restructure  in  2010.  The  Air  Force’s  Evolved 
Expendable  Launch  Vehicle  (EELV)  breached  in  May  2012,  exceeding 
its  original  projected  per-unit  cost  by  58  percent. 

Recent  DoD  guidance  and  oversight  suggest  a  need  for  more  focused 
investment  of  funds  to  promote  better  design  decisions  and  perfor¬ 
mance  outcomes  in  acquisition.  DoD  Instruction  5000.02  (2008)  now 
requires  a  materiel  development  decision  before  a  program  enters  the 
acquisition  process.  As  a  result,  programs  must  invest  more  funds  in 
alternative  design  analysis  before  entering  development  or  production. 
WSARA  emphasizes  investment  in  competitive  prototyping  and  trade¬ 
off  analysis  before  development  starts.  A  2012  GAO  report  (GAO,  2012a, 
p.  2)  and  the  Better  Buying  Power  memorandum  emphasize  investing  in 
design,  development,  and  production  to  mitigate  performance  failure. 
Maintaining  competition  throughout  the  system  life  cycle  by  investing 
in  alternative  products  can  help  meet  the  government’s  cost,  schedule, 
and  performance  goals,  and  at  the  same  time  expand  the  marketplace 
of  qualified  firms. 
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The  Shrinking  Competitive  Marketplace 

Following  the  large  increase  in  defense  acquisition  dollars  dur¬ 
ing  Operation  Iraqi  Freedom,  the  DoD  now  faces  a  period  of  declining 
budgets.  This  pattern  was  last  seen  at  the  Reagan-era  defense  buildup 
in  the  1980s  and  the  drawdown  at  the  conclusion  of  the  Cold  War  in  the 
1990s.  Both  patterns  were  marked  by  the  significant  consolidation  of  the 
defense  industrial  base,  making  acquisition  through  competition  more 
difficult  to  realize. 

Many  capable  firms  competed  for  contracts  in  the  1980s,  and  dual 
sourcing  was  used  as  an  acquisition  strategy  for  several  weapon  sys¬ 
tems.  Today,  dual  sourcing  is  rare  in  weapon  systems’  procurements 
because  the  marketplace  contains  fewer  suppliers  and  the  DoD  has  less 
investment  dollars  to  develop  a  second  source.  The  top  four  contrac¬ 
tors  delivering  defense  products— Lockheed  Martin,  Boeing,  General 
Dynamics,  and  Raytheon— have  not  changed  from  1999  to  2010  (Ellman 
et  al„  2011,  p.  27). 

In  an  interview  with  Reuters  in  2011,  Deputy  Secretary  of  Defense 
Ashton  Carter  expressed  concern  about  another  round  of  mergers  in  the 
defense  contractor  base,  emphasizing  the  need  for  a  strong,  technically 
vibrant  defense  industry  (Carter,  2011).  He  suggested  several  strategies 
to  nurture  such  an  industry,  such  as  increased  attention  to  more  dynamic 
lower  tier  companies  and  greater  opportunity  for  entry  of  smaller  firms 
and  start-ups. 

In  Fiscal  Year  (FY)  2011,  the  DoD  reported  that  it  spent  a  total  of 
$374  billion  on  contracts  for  products  and  services  and  research,  and 
awarded  58  percent  of  its  contracts  competitively  (GAO,  2012b,  p.  8). 
However,  products  alone  represented  only  41  percent  of  competitive 
bidding,  and  major  weapon  systems  accounted  for  a  large  part  of  this 
number.  The  DoD  had  96  MDAPs  in  2011,  with  an  overall  2011  develop¬ 
ment  and  production  cost  of  $170  billion. 

The  following  review  of  several  defense  products  illustrates  how 
competitive  practices  have  shifted  since  the  1980s. 

Contracts  for  aircraft  are  competitively  awarded  to  a  single  bidder, 
with  little  evidence  of  continued  competition  during  the  program  life 
cycle.  This  approach  has  persisted  since  the  1980s,  largely  because  of 
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proprietary  data,  high  barriers  to  entry,  and  an  unwillingness  to  change 
contractors  once  a  design  decision  has  been  made.  The  number  of  com¬ 
petitors  has  decreased  drastically  since  the  sharp  reduction  in  defense 
spending  during  the  mid-1990s.  Today,  only  a  few  companies  can  vie  for 
aircraft  contracts,  and  the  competitions  most  often  result  in  a  single 
award.  For  example,  the  Air  Force  recently  made  a  hotly  contested  (and 
controversial)  single  award  for  replacement  of  the  KC-135  tanker,  cover¬ 
ing  production  of  179  aircraft  at  a  cost  of  $35  billion;  and  the  Navy  P-8 
Poseidon  is  being  developed  and  produced  under  a  single  award  for  117 
aircraft.  Even  though  GAO  found  that  more  than  half  of  aircraft  pro¬ 
grams  are  known  to  experience  design  problems  and  cost  growth  (GAO, 
2011b,  p.  11),  neither  program  involves  competition  from  an  alternate 
source  or  lower  tier  company. 

Similarly,  contracts  to  build  ships  are  competitively  awarded  to  a 
single  contractor  because  of  the  high  investment  in  infrastructure  and 
low  production  volume.  In  the  1980s,  ship  production  was  high  and  six 
commercial  shipbuilders  accommodated  the  world  reach  of  a  planned 
600-ship  Navy.  Today,  the  fleet  is  smaller  (fewer  than  300  ships),  and  only 
two  major  commercial  shipyards  (General  Dynamics  and  Huntington- 
Ingalls)  survive  within  the  defense  industrial  base.  Shipbuilding  has 
suffered  extensive  program  stretch-out  due  to  reduced  budgets  and 
cost  overruns.  Aircraft  carriers  take  4  years  to  complete;  each  one  (e.g., 
the  new  Gerald  Ford  Class)  costs  $12  billion  to  develop  and  $14  billion 
to  build.  Given  that  level  of  investment  in  time  and  money,  it  would  be 
difficult  to  engage  a  second  shipyard  to  provide  continuous  competition. 

Historically,  engine  development  and  production  have  been  highly 
competitive,  due  to  the  high  volume  and  advancing  propulsion  technol¬ 
ogy  in  both  the  commercial  and  military  market.  During  the  1980s,  dual 
sourcing  was  used  very  successfully  in  this  market.  The  “engine  wars” 
for  multiple  military  aircraft  drove  prices  down  and  introduced  much- 
needed  improvements  in  power  and  fuel  efficiency.  For  example,  in  the 
1980s  the  Air  Force  added  General  Electric  as  a  second  source  with  Pratt 
&  Whitney  for  the  F-15  and  F-16  fighter  engines,  saving  $2-3  billion  over 
the  20-year  life  cycle  and  doubling  the  reliability-per-1000-engine- 
flight-hours  (Gansler,  Lucyshyn,  &  Arendt,  2009,  p.  27).  Each  year,  the 
award  for  the  production  quantity  was  split  between  the  two  sources. 
The  additional  benefit  during  the  1980s  was  that  GE  invested  its  own 
money  to  develop  and  deliver  its  engines.  In  the  21st  century,  contracts 
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for  engines  have  moved  away  from  this  dual-source  environment.  The 
DoD  decided  to  cancel  the  second  source  for  the  F-35  JSF  because  the 
development  costs  were  too  high  ($2.9  billion). 

According  to  various  studies,  dual-sourcing  for  14  tactical  missile 
programs  between  1975  and  1995  saved  20  percent  over  the  life  cycle. 
The  Navy  Tomahawk  missile  made  annual  split  awards,  saving  over 
$270  million  and  improving  performance  reliability  from  80  percent 
to  97  percent.  The  DoD  made  only  a  minor  investment  to  develop  the 
second  source  (2  percent  of  production  costs)  (Gansler  et  al.,  2009,  p. 
30).  Today,  missile  programs  have  returned  to  single  awards,  and  many 
experience  design  issues  and  cost  overruns.  The  cost  of  the  Army  Patriot 
PAC-3  increased  by  77  percent,  from  about  $3.9  billion  in  1994  to  about 
$6.9  billion  in  March  2000,  due  to  design  issues  and  program  stretch-out 
(GAO,  2011b,  p.  14).  The  Pentagon’s  Missile  Defense  Agency  (MDA)  has 
spent  nearly  $56  billion  researching  and  deploying  various  systems  since 
2002;  GAO  estimates  that  MDA  contractors  overran  budgeted  costs  by 
$152.4  million  in  FY08. 

Subsystems  and  components  have  also  had  success  with  continu¬ 
ous  competition.  The  Joint  Direct  Attack  Munition  (JDAM)  program 
conducted  dual-sourcing  competitions  in  the  late  1990s  and  2000s  to 
effectively  control  unit  costs  (Meyers,  2002).  The  Navy  has  used  mul¬ 
tiple  sources  for  its  sonobouys  since  the  1980s,  but  now  awards  are 
made  to  single  entities.  Recently,  the  Navy  awarded  a  major  contract  for 
over  50,000  passive  sonobuoys  to  ERAPSCO,  a  joint  venture  of  Sparton 
Corporation  and  Ultra  Electronics  (USSI)  (Sparton  Corporation,  2013). 
In  December  2011,  the  Navy  had  awarded  a  $25  million  contract  to  Signal 
Systems  Corporation  under  the  Small  Business  Innovation  Research 
(SBIR)  program  to  explore  new  underwater  measurements,  which  could 
provide  an  alternative  to  the  sonobuoys.  During  2004  to  2006,  the  U.S. 
Army  Forward  Looking  Infrared  Program  Office  considered  bringing 
in  additional  sources  to  motivate  the  primary  manufacturer  to  build  a 
limited  number  of  prototypes.  Although  the  prototypes  never  went  into 
production,  the  effort  did  provide  leverage  to  control  the  technology  and 
price  in  subsequent  design  and  production  units. 
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Competitive  Acquisition  Approaches 

The  Federal  Acquisition  Regulation  (FAR)  and  the  Defense  Federal 
Acquisition  Regulation  Supplement  (DFARS)  contain  regulatory  and 
policy  guidance  for  implementing  competitive  acquisitions.  With  limited 
exceptions,  contracting  officers  are  required  to  promote  “full  and  open 
competition”;  that  is,  ensure  that  all  responsible  sources  may  compete. 

Procurement  Methods  That  Support  Competition 

The  FAR  recognizes  several  competitive  acquisition  methods. 
Awards  are  made  to  a  single  winner  or  multiple  sources,  thereby  influ¬ 
encing  the  presence  of  competition  for  the  remainder  of  the  life  cycle. 
Dual  sourcing  and  leader-follower  are  two  established  acquisition  meth¬ 
ods  used  to  implement  continuous  competition  throughout  the  life  cycle. 
DFARS  (subpart  207.1)  recognizes  dual  sourcing  as  a  viable  approach 
to  acquisition.  This  method  creates  competitive  pressure  through  hav¬ 
ing  two  or  more  sources  deliver  systems  that  meet  requirements.  Dual 
sourcing  has  been  used  primarily  by  programs  with  reasonable  start-up 
costs  that  produce  large  quantities  of  an  item  at  the  least  total  cost.  Under 
leader-follower  sourcing,  described  in  FAR  (subpart  17.4),  an  otherwise 
sole-source  “leader”  contractor  provides  “assistance  and  know-how”  to 
a  “follower”  contractor  to  achieve  the  benefits  of  multisourcing. 

DoD  agencies  use  several  other  techniques  to  bring  alternative  prices 
to  the  development  or  production  phase  of  a  life  cycle.  Military  depots 
use  both  organic  and  contracted  industry  services  for  maintenance  and 
repair  to  provide  a  dual  source  and  alternative  pricing.  Government 
laboratories  offer  competitive  cost  comparisons  by  evaluating  technology 
against  industry  designs  and  validating  costs  for  Engineering  Change 
Proposals  (ECPs).  DoD  uses  independent  verification  and  validation  con¬ 
tracts  to  conduct  independent  reviews  of  aspects  of  a  program  and  give 
the  acquiring  organization  leverage  in  negotiating  with  potential  con¬ 
tractors.  The  SBIR  program  can  add  value  by  exploring  new  technology 
and  costs.  The  DoD  Broad  Agency  Announcements  call  for  alternative 
source  data  for  better  engineering  design  decisions  and  cost  estimating. 


Defense ARJ,  April  2013,  Vol.  20 No.  1: 027-058 


46 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


Commercial  competitive  development  model.  This  open-mar¬ 
ket  strategy  encourages  all  contractors  to  develop  products  at  their  own 
cost.  The  government  has  the  option  to  buy  these  products  at  a  per-unit 
cost  once  the  items  are  fully  developed  and  ready  for  production.  Firms 
are  willing  to  fund  the  development  if  they  believe  the  government  will 
choose  to  buy  their  products  at  a  price  and  quantity  that  enables  them 
to  recoup  costs  and  earn  a  reasonable  profit  in  the  production  phase. 
This  approach  is  best  suited  to  information  technology  systems  that 
allow  contractors  to  develop  applications  on  an  existing  infrastructure. 
However,  it  can  also  be  used  in  developing  components  on  top  of  open 
hardware  platforms.  For  instance,  airframes,  ships,  and  vehicle  classes 
present  a  standard  platform,  but  competition  could  occur  for  the  various 
subsystems  (e.g.,  avionics,  navigation,  and  fire  control  systems). 

Competitive  orders  (indefinite  delivery/indefinite  quantity). 

The  government  awards  contracts  to  multiple,  qualified  contractors 
to  meet  a  broad  set  of  requirements.  The  government  negotiates  pric¬ 
ing,  terms,  and  conditions  with  each  vendor.  The  multiple  awardees 
vie  for  task/delivery  orders  in  a  post-award  competitive  environment, 
keeping  competitive  pressures  in  play  throughout  the  life  of  a  contract. 
This  strategy  works  best  when  requirements  can  be  broken  into  several 
manageable  tasks  that  different  contractors  can  perform  independently 
over  a  period  of  time. 

Competitive  dual  sources.  The  government  fully  funds  two 
contractors  to  execute  their  designs  or  solutions  to  meet  a  need.  The 
contractors  fully  develop  and  produce  their  designs,  thus  providing  the 
government  with  two  viable  solutions.  The  two  sources  continuously 
drive  down  prices  while  also  improving  the  performance  and  reliability 
of  their  products  over  time.  Of  the  continuous  competition  strategies, 
this  approach  requires  the  greatest  up-front  investment  by  the  govern¬ 
ment,  but  it  also  creates  the  most  competition  and  the  highest  probability 
of  meeting  program  mission  needs  on  schedule. 

Competitive  multisourcing  with  distributed  awards.  Under 
this  new  approach,  the  government  awards  contracts  to  two  (or  more) 
sources,  with  a  primary  contractor  receiving  the  majority  of  funding.  A 
second  contractor  is  selected  to  create  a  continuous  competitive  envi¬ 
ronment  and  to  provide  a  viable  backup  should  the  primary  contractor 
fail  to  meet  program  objectives.  The  next  section  of  this  article  explores 
this  approach  in  greater  detail. 
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Competitive  Multisourcing  with 
Distributed  Awards 

Competitive  Multisourcing  with  Distributed  Awards  offers  an 
alternative  to  full  dual  sourcing,  enabling  the  government  to  maintain 
multiple  viable  sources  without  having  to  “fully”  fund  or  “share”  work 
among  competitors.  Under  this  approach,  the  second  contractor  does 
not  deliver  an  equal  share,  but  receives  sufficient  funds  to  mature  an 
alternative  design  and  bring  competitive  pressure  into  the  environment. 
This  can  provide  the  government  with  a  viable  alternative  contractor  if 
the  prime  underperforms. 

Definitions 

Under  this  model,  the  government  awards  the  majority  of  funding  to 
a  prime  contractor,  and  at  the  same  time  provides  a  smaller  amount  of 
funding  to  a  secondary  source.  Keeping  a  second  source  under  contract 
at  even  a  low  level  (e.g.,  5-10  percent  of  prime  contract  costs)  maintains 
significant  competitive  pressure  on  the  prime  contractor  by  greatly 
reducing  the  barriers  of  entry  into  the  program  (i.e.,  it  lowers  the  costs 
of  switching  if  the  prime  does  not  perform  satisfactorily).  It  also  allows 
the  second  source  to  refine  and  mature  its  technical  approach  and  gain 
familiarity  with  the  program’s  operations.  The  cost  of  implementing  this 
competitive  multisourcing  approach  can  be  relatively  small  compared  to 
the  benefits  of  competition  that  it  provides. 

The  DoD  can  apply  this  approach  in  several  ways  to  maintain  con¬ 
tinuous  competition  in  all  stages  of  the  acquisition  life  cycle. 

Percentage-based  distributions  Under  this  strategy,  a  set  per¬ 
centage  of  funding  is  allocated  to  each  source.  For  example.  Vendor  A 
submits  the  best  offer  and  receives  the  majority  of  funding  (e.g.,  90  per¬ 
cent)  as  the  primary  source.  Vendor  B  submits  the  second-best  offer  and 
receives  a  smaller  percentage  of  funding  (e.g.,  10  percent)  to  partially 
develop  its  design  or  to  work  on  a  particular  subset  of  the  contract  require¬ 
ments.  This  strategy  keeps  a  second  viable  source  in  play  during  the 
prototyping,  development,  production,  and  sustainment  phases,  which 
will  provide  competitive  pressure  to  motivate  the  primary  contractor. 
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Full  development  with  scaled  production.  Under  this  strategy, 
two  or  more  contractors  are  fully  funded  to  develop  prototype  products. 
After  the  two  prototypes  have  been  delivered,  the  government  selects  one 
contractor  for  full-scale  production  and  awards  a  contract  for  limited 
production  to  the  second  source.  This  strategy  can  work  best  for  products 
to  minimize  risk  during  the  design  phase  of  the  program. 


Next  increment  prototype  model  Under  this  strategy,  the  DoD 
uses  a  primary  source  to  maintain  engineering  capability  for  the  current 
production  unit.  A  lesser  amount  of  funding  is  provided  to  a  secondary 
source  to  build  a  prototype  for  the  next  program  increment.  In  addition 
to  getting  a  head  start  on  the  next  spiral  of  development,  this  mechanism 
allows  the  DoD  to  introduce  a  second  capable  source  and  position  it  to 
compete  with  the  prime  for  the  next  program  increment. 
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Partial  contractor-funded  development  model.  Under  this 
strategy,  the  DoD  caps  the  amount  of  development  funding  to  a  second 
contractor  (e.g.,  30  percent  of  proposed  costs).  The  contractor  has  the 
option  to  fully  fund  the  development  of  the  proposed  design.  This  gives 
the  contractor  the  potential  to  recapture  these  development  costs  during 
the  production  phase  if  the  government  selects  the  second  contractor’s 
design  for  production. 

Conditions  for  Use  of  Competitive  Multisourcing  with 
Distributed  Awards 

Certain  conditions  favor  the  successful  application  of  Competitive 
Multisourcing  with  Distributed  Awards  during  development  of  contract¬ 
ing  strategies  for  acquisition  programs.  These  conditions  are  derived 
from  the  historical  perspective  and  lessons  learned  addressed  earlier 
in  this  article. 

High  quantities  with  economic  production  rates  This  con¬ 
dition  can  apply  in  both  development  and  production  phases  of  the 
acquisition.  Competition  can  be  maintained  in  the  production  phase  in 
situations  where  investment  costs  are  low,  production  accounts  for  the 
majority  of  the  costs,  and  contractors  go  head-to-head  for  high-volume 
returns.  Maintaining  a  second  source  in  the  development  phase  will  work 
best  when  the  government  declares  the  intent  to  maintain  dual  sources 
in  production. 

Credible  competition.  The  second  source  must  represent  effective 
leverage  and  alternatives  to  the  single-source  environment.  This  situa¬ 
tion  can  occur  in  an  environment  where  industry  competes  on  a  regular 
basis,  and  the  prime  contractor  recognizes  the  second  source  as  a  peer 
competitor.  The  contracting  arrangement  must  also  facilitate  alternat¬ 
ing  from  one  source  to  the  other. 

Sufficient  technical  knowledge  in  industry.  Both  the  prime  and 
secondary  source  must  already  have  enough  knowledge  and  intellectual 
property  to  offer  credible  competitive  products.  At  the  very  least,  the 
secondary  source  must  have  adequate  technical  and  manufacturing 
readiness  to  be  viewed  as  legitimate  competition.  The  contract  and 
program  reporting  mechanism  must  track  the  costs  of  both  competitors 
in  order  to  close  the  design  maturity  gap  and  improve  the  Technology 
Readiness  Level  (TRL). 
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Effective  cost-benefit  analysis.  While  it  may  cost  5-10  percent 
of  the  program  budget  in  the  short  term,  in  depth  cost-benefit  analysis 
has  the  potential  to  save  far  more  over  the  long  term.  The  analysis  can 
consider  items  such  as  reduced  barriers  to  program  entry,  lower  costs  for 
switching  between  contractors,  and  the  benefits  of  technology  develop  - 
ment  and  design  maturity.  The  business  case  must  also  include  budget 
and  schedule  considerations.  Executing  an  acquisition  strategy  and 
keeping  a  second  source  in  the  competition  increase  the  likelihood  that 
the  prime  contractor  will  perform  closer  to  budget  and  schedule. 

Challenges 

The  WSARA,  “Better  Buying  Power”  tenets,  and  GAO  reports  rec¬ 
ommend  prototyping  and  maturing  designs  in  a  price-competitive 
environment.  An  acquisition  strategy  of  this  kind  does  bring  several 
challenges.  First,  the  DoD  has  declining  discretionary  funding.  However, 
investing  in  alternative  sources  will  allow  better  design  and  technical 
decisions  during  the  program  life  cycle.  Second,  writing  an  additional 
contract  for  the  second  source  creates  an  administrative  burden  for  the 
contracting  staff.  Still,  the  ability  to  control  engineering  design  changes 
can  mitigate  the  risk  of  Nunn-McCurdy  program  breaches,  which  can 
impose  inordinate  administrative  costs  on  a  program  office.  Third,  most 
program  offices  prefer  to  stay  with  the  same  contractor  rather  than  incur 
the  time  and  expense  of  new  competitions.  A  second  source  could  be  a 
controlling  factor  for  cost  growth  and  schedule  creep.  The  largest  hurdle 
to  overcome  is  creating  an  industrial  environment  that  encourages 
competitive  forces  to  invest  in  this  acquisition  approach.  The  DoD  can 
meet  this  challenge  by  providing  the  potential  for  increased  business  in 
previously  closed  markets. 

However,  the  challenges  do  not  involve  only  cost,  budget,  and 
resource  constraints.  The  government  must  take  various  actions  to 
minimize  the  risks  of  adopting  this  new  and  innovative  method  of  con¬ 
tinuous  competition. 
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Ensure  the  Second  Source  Poses  an  Actual  Threat  to  the 
Prime  Source 

With  the  shrinking  industrial  base,  only  a  handful  of  companies  now 
compete  for  major  weapon  systems.  The  second  source  must  be  credible 
and  known  in  the  market.  The  government  can  identify  a  viable  second 
source  in  various  ways,  such  as  by  evaluating  commercial  products  or 
considering  a  company’s  level  of  independent  research  and  development 
to  complement  government  investments. 

Coordinate  the  Activities  Between  the  Two  Sources 
to  Ensure  Delivery  of  Products  and  Solutions  at  Set 
Program  Milestones 

Both  contractors  must  be  effectively  evaluated  along  the  same  path 
or  according  to  the  same  milestones,  taking  their  different  solutions  into 
account.  The  challenge  is  to  coordinate  activities  for  both  contractors, 
such  as  testing,  to  ensure  synchronization. 

Close  the  Maturity  Gap  at  the  Same  Rate  and  TRL 

The  second  source  should  have  a  relatively  mature  product,  or  should 
invest  in  developing  such  technology  on  its  own.  Any  newcomer  to  the 
industry  or  the  market  will  experience  technology  lag  and  will  therefore 
not  pose  any  competitive  threat  to  the  prime  contractor. 

Recognize  and  Accept  the  Fact  That  Cost  Savings  Will 
Not  Be  Realized  Until  the  Production  Phase 

A  business  case  is  critical  to  identify  the  long-term  benefits  of  compe¬ 
tition.  It  may  be  difficult  to  clearly  articulate  savings  without  empirical 
data,  but  benchmarking  historical  successes  and  establishing  clear 
measures  for  technology  maturity  levels  will  help  quantify  the  benefits. 

Recognize  Supply  Chain  Risks 

Multiple  prime  sources  can  impact  the  number  and  type  of  suppliers. 
Ensuring  that  there  is  a  sufficient  and  security-assured  supply  chain  for 
the  prime  and  the  second  source  will  be  critical  to  uninterrupted  delivery 
in  a  competitive  environment. 
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Benefits  and  Measures  of  Success 

In  this  period  of  fiscal  austerity,  funding  a  second  source  may  appear 
to  be  a  luxury.  To  maximize  the  advantages  of  continuous  life-cycle  com¬ 
petition,  the  DoD  needs  to  adopt  a  dual  or  multiprovider  strategy  from 
the  outset  of  the  program  planning  process.  While  continuous  competi¬ 
tion  approaches  may  require  greater  up-front  funding,  they  have  the 
potential  to  save  far  more  over  the  long  term  and  to  provide  additional, 
nonmonetary  program  benefits. 

Benefits 

Dual  sourcing  has  already  proven  effective  in  reducing  costs,  as  evi¬ 
denced  by  the  engine  wars,  referenced  earlier.  According  to  a  2001  study 
by  the  RAND  Corporation  (Birkler  et  al.,  2001,  p.  16),  the  introduction 
of  a  second  source  during  the  production  of  the  Tomahawk  missile  led 
to  estimated  savings  of  $630  million,  while  improving  the  missile’s  reli¬ 
ability  from  approximately  80  percent  to  97  percent.  The  same  study  also 
revealed  that  the  10  DoD  aircraft  programs  that  involved  no  competition 
during  the  production  phase  experienced  an  average  46  percent  increase 
in  cost  over  the  original  budget. 

Continuous  competition  places  competitive  pressure  on  the  prime 
contractor  through  the  presence  of  alternative  sources.  The  most  sig¬ 
nificant  benefits  include: 

Allowing  a  second  source  to  refine  and  mature  its  technical 
design.  The  WSARA,  GAO,  and  DoD  all  recognize  the  need  for  better 
investments  in  engineering  design  and  technology  to  improve  decision 
making  in  choosing  a  contractor.  Furthermore,  problems  with  engi¬ 
neering  design  caused  the  majority  of  Nunn-McCurdy  breaches.  The 
additional  design  maturity  would  produce  alternative  cost  estimates 
for  ECPs  to  enable  better  cost  analysis  and  cost  containment  (GAO, 
2011b,  p.  3). 

Maintaining  the  pressure  of  price  competition.  The  FAR 

addresses  the  critical  nature  of  competitive  environments  and  the 
importance  of  maintaining  the  spirit  of  competition  throughout  the  life 
cycle.  Distributed  competition  would  create  an  environment  of  competi¬ 
tion  for  the  prime  manufacturer,  even  though  that  competition  would  not 
be  at  the  full,  traditional,  dual-source  production  level. 
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Having  a  viable  second  source  available  as  insurance  for 
transition  in  case  of  performance  failure  by  the  prime.  A  second 
contractor,  if  perceived  as  a  viable  alternative,  can  prevent  monopolis¬ 
tic  behavior  by  the  primary  contractor.  The  average  time  to  compete  a 
major  program  is  12  to  18  months,  and,  quite  often,  that  does  not  include 
the  critical  transition  time  necessary  for  the  new  contractor  to  become 
fully  functional. 

Measures  of  Success 

Recent  program  reviews  by  DoD  and  GAO  highlighted  three  main 
areas  of  concern:  cost  growth,  schedule  slip,  and  performance  fail¬ 
ure.  The  government  could  determine  the  success  of  Competitive 
Multisourcing  with  Distributed  Awards  by  measuring: 

•  The  ability  to  contain  costs,  measured  against  statisti¬ 
cal  cost  growth  percentages  over  the  life  cycle  of  selected 
programs,  and  benchmarked  with  CSIS,  GAO,  and  Nunn- 
McCurdy  cost  figures. 

•  The  ability  to  reduce  known  causes  for  schedule  slips  in 
production,  such  as  a  lack  of  alternate  sources  of  critical 
suppliers  or  unplanned  engineering  changes. 

•  The  ability  to  improve  performance  by  achieving  or  exceed¬ 
ing  technical  performance  against  key  performance 
parameters  that  are  a  part  of  program  requirements. 

Implementation 

The  DoD  can  apply  the  following  guidelines  in  adopting  Competitive 
Multisourcing  with  Distributed  Awards  as  an  alternative  to  standard 
dual  sourcing: 

•  Apply  the  method  under  a  clear  set  of  conditions  and  using 
a  business  case  where  the  value  of  an  additional  source  will 
improve  performance  and  control  costs. 

•  Evaluate  the  cost  of  the  additional  source  as  an  investment 
to  improve  decision  making  and  enhanced  life-cycle  cost 
estimating. 
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•  Develop  risk/reward  factors  that  clearly  incentivize  both 
the  prime  and  the  second  source  contractor. 

•  Include  clauses  in  the  contract  specifically  to  accommodate 
technology  sharing  and  ease  of  transition  from  one  contrac¬ 
tor  to  another. 

•  Engage  industry  through  clear  direction  and  defined 
outcomes. 

The  DoD  should  consider  Competitive  Multisourcing  with 
Distributed  Awards  for  the  following  products  and  programs: 

Products  with  Known  Dual  Sourcing  Success 

In  the  past,  dual  sources  have  achieved  success  in  developing  and 
producing  engines  because  of  the  large  volume  and  economic  produc¬ 
tion  rates.  High  costs  prevented  the  F-35  JSF  program  from  pursuing 
a  second-engine  source.  This  program  would  be  a  good  candidate  for 
Competitive  Multisourcing  with  Distributed  Awards,  allowing  the  Air 
Force  to  retain  a  second  contractor  at  a  lower  level  to  mature  the  design 
and  become  a  potential  production  source. 

Products  in  Highly  Competitive  Environments  That 
Resulted  in  A  Single  Award 

The  recent  competition  for  the  Air  Force  tanker  resulted  in  a  single 
award.  The  selected  solution  was  rated  at  TRL  6,  so  further  design  and 
development  will  occur  before  full  production.  Keeping  a  second  source 
in  play  would  leverage  technology  brought  to  the  table  during  the  com¬ 
petition  and  could  enable  higher  TRL  levels  at  a  faster  pace. 

Programs  with  High  Cost  Overruns 

The  EELV  is  under  Nunn-McCurdy  review  because  of  skyrocketing 
costs.  The  Air  Force  plans  to  introduce  competition,  currently  under  a 
single-awardjointventure  contract,  by  2018  (Butler,  2011,  p.  4).  Applying 
Competitive  Multisourcing  with  Distributed  Awards  to  this  program 
with  a  low-level  second  source  would  allow  development  of  additional 
technology  and  would  contain  costs. 
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Recommendations 

In  1954,  President  Dwight  D.  Eisenhower’s  Secretary  of  Defense, 
Charles  Erwin  Wilson,  notably  used  the  phrase  “bigger  bang  for  the 
buck”  to  capture  the  concept  of  greater  worth  for  the  money  spent.  The 
phrase  perfectly  summarizes  the  benefits  of  continuous  competition 
and  Competitive  Multisourcing  with  Distributed  Awards.  DoD  should 
adopt  Competitive  Multisourcing  with  Distributed  Awards  as  an  alter¬ 
native  method  for  continuous  competition  and  dual  sourcing.  The  DoD 
should  redefine  competition  so  that  it  is  no  longer  viewed  merely  as  an 
up-front  activity  limited  to  the  contracting  process.  Applying  continuous 
competition  to  the  right  set  of  DoD  acquisition  programs  could  replicate 
proven  successes  at  a  far  broader  scale,  yielding  significant  benefits  to 
our  nation’s  warfighters  as  well  as  to  the  program  offices  that  deliver 
capabilities  to  them. 
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to  Maximize  the  Probability 
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|  James  Bradshaw  and  Su  Chang 


The  federal  contracting  process  should  enable  a 
government  organization  to  select  a  contractor  that  will 
become  a  true  business  partner.  Today’s  source  selec¬ 
tion  processes  evaluate  how  well  a  contractor  proposes 
a  solution;  however,  the  government’s  processes  are 
ill  suited  to  evaluate  how  well  a  contractor  can  deliver 
on  its  proposal.  The  Department  of  Defense  (DoD) 
relies  too  heavily  on  the  contractor’s  proposal  versus 
evaluating  past  performance.  The  lack  of  past  perfor¬ 
mance  data  and  processes  to  evaluate  companies’ 
qualifications  has  contributed  to  program  failures,  cost 
overruns,  and  schedule  delays.  Without  adequate  data 
and  processes,  the  DoD  increases  its  risk  of  duplicating 
previous  program  failures  and  misses  the  opportunity  to 
capture  this  information,  thereby  preventing  repeated 
mistakes  with  the  same  contractor. 
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Selecting  the  right  contractor  is  perhaps  the  most  critical  factor  in 
achieving  future  program  success.  This  research  identifies  processes  and 
proposes  strategies  that  could  strengthen  how  the  government,  specifi¬ 
cally  the  DoD,  selects  an  industry  partner  to  maximize  the  probability 
of  program  success. 

Ideally,  the  relationship  between  the  government  and  a  contractor 
should  be  a  genuine  business  partnership.  When  awarding  a  contract  for 
a  major  weapon  system  or  program,  especially  one  that  requires  signifi¬ 
cant  development,  the  DoD  must  expect  the  relationship  to  be  a  long-term 
commitment.  The  contractor  and  government  program  management 
office  (PMO)  each  play  numerous  critical  roles  throughout  the  program 
planning,  executing,  and  reporting  cycles.  In  a  successful  program,  a 
contractor  organization  and  the  PMO  work  in  harmony  to  strive  toward 
program  success.  They  understand  and  leverage  each  other’s  strengths; 
they  communicate  frequently;  they  collectively  manage  the  vast  web  of 
program  stakeholders;  and  they  collectively  establish  the  reputation  and 
credibility  of  the  program. 

The  government  must  therefore  strive  to  enter  into  business  relation¬ 
ships  that  provide  high  confidence  of  success.  Selecting  an  experienced 
and  qualified  business  partner  is  a  critical  step  in  achieving  this  objec¬ 
tive.  While  evaluation  of  the  contractor’s  proposal  is  important,  this 
review  can  provide  only  a  prediction  of  what  may  happen  and  a  descrip¬ 
tion  of  how  the  contractorpZansto  perform.  By  contrast,  the  contractor’s 
experience  and  past  performance  offer  an  objective  measure  of  what  the 
firm  has  actually  accomplished  in  the  past. 

The  article  identifies  three  key  problem  areas: 

1.  The  DoD  evaluates  the  contractor’s  proposed  solution 
rather  than  the  contractor’s  record  of  actual  performance. 

During  the  source  selection  process,  the  DoD  evaluates  a  con¬ 
tractor’s  proposal,  which  consists  of  cost,  technical,  and  past 
performance  volumes.  In  most  cases,  however,  past  performance 
has  little  or  no  impact  on  the  final  selection  of  a  contractor. 
Instead,  this  aspect  is  often  minimized  as  a  superficial  pass/ 
fail  test  rather  than  being  viewed  as  an  effective  tool  to  pre¬ 
dict  how  well  a  contractor  will  perform  in  the  future.  Without 
objective  information  on  past  performance  and  processes  to 
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evaluate  contractor  qualifications,  DoD  is  reluctant  to  discrimi¬ 
nate  among  proposals  based  on  past  performance,  especially 
given  the  risk  of  protest. 

2.  The  DoD  lacks  consistent  and  thorough  processes  to  evalu¬ 
ate  key  personnel.  The  personnel  on  the  contractor  program 
team  make  vital  contributions  to  program  success,  and  as  such, 
their  qualifications  should  be  an  important  evaluation  factor. 
Contractors  generally  identify  key  personnel,  and  include  their 
resumes  as  part  of  the  proposal  package.  However,  most  source 
selections  limit  evaluation  of  these  key  personnel  to  a  simple 
determination  of  whether  the  proposed  team  members  meet  the 
minimum  experience  criteria. 

3.  The  DoD  lacks  adequate  tools  to  collect,  analyze,  and 
report  past  performance  information  consistently  across 
contracts.  The  ability  to  evaluate  past  performance  effectively 
depends  heavily  on  the  availability  of  tools  and  the  quality  of  data 
content.  The  government  is  currently  improving  and  consolidat¬ 
ing  its  repositories  of  past  performance  data;  however,  the  DoD 
lacks  capabilities  for  information  collection  and  has  not  consis¬ 
tently  enforced  data  entry  requirements. 

These  problem  areas  represent  three  different  deficiencies  in  the 
evaluation  of  a  contractor’s  past  performance.  Evaluation  is  hampered 
by  the  lack  of  adequate  tools  to  collect,  analyze,  and  report  relevant 
information.  The  lack  of  processes  to  assess  the  capabilities  and  expe¬ 
rience  of  key  personnel  puts  at  risk  the  government’s  ability  to  predict 
future  contract  performance.  In  addition,  the  ability  to  collect,  analyze, 
and  report  information  on  past  performance  is  impaired  by  the  lack  of 
policies  and  processes  to  incentivize  better  reporting.  Together,  these 
shortcomings  affect  the  overall  contracting  process,  and  contribute 
to  a  cycle  of  inadequacies  that  has  resulted  in  the  numerous  contract¬ 
ing  challenges  we  face  today.  The  DoD  needs  to  take  a  comprehensive, 
coherent  approach  to  resolve  these  problems  to  achieve  overall  past 
performance  improvements. 
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Background 

With  the  passage  of  the  Federal  Acquisition  Streamlining  Act  of 
1994,  Congress  acknowledged  that  it  is  appropriate  and  relevant  for  the 
government  to  consider  a  contractor’s  past  performance  in  evaluating 
whether  that  contractor  should  receive  future  work  (DoD,  2008,  p.  1). 
In  response,  federal  departments  initiated  procedures  and  systems 
to  record  and  use  information  on  past  contractor  performance  during 
source  selection.  Although  the  government  recognizes  that  system¬ 
atically  documenting  the  contractor’s  performance  becomes  a  powerful 
motivator  for  a  contractor  to  sustain  high-quality  outcomes,  consistent 
processes  for  recording  performance  have  been  difficult  to  instantiate 
and  maintain  (Office  of  Federal  Procurement  Policy  [OFPP],  2000,  p.  1). 

In  May  2010,  the  OFPP  directed  government  agencies  to  integrate 
past  performance  data  systems  to  provide  consistency  and  ensure 
reliability  of  data  across  the  federal  government.  The  Acquisition 
Committee  for  eGover  nment  determined  that  the  entire  federal  govern¬ 
ment  will  use  the  DoD  Contractor  Performance  Assessment  Reporting 
System  (CPARS)  as  the  single  system  for  collecting  and  transmitting 
performance  evaluations  to  the  Past  Performance  Information  Retrieval 
System  (PPIRS,  n.d.,  p.  1).  The  Navy  created  CPARS  in  1998  to  meet  infor¬ 
mation  requirements  established  by  the  Federal  Acquisition  Regulation 
(FAR).  Due  to  inconsistent  CPARS  data  collection  and  reporting,  the 
DoD  published  a  best  practices  guide  titled  A  Guide  to  Collection  and 
Use  of  Past  Performance  Information  in  May  2003.  In  June  2007,  the 
DoD  issued  another  CPARS  Policy  Guide  that  required  all  new  contracts 
within  a  certain  dollar  threshold  to  register  in  CPARS  within  30  days  of 
contract  award  (DoD,  2008,  p.  5). 

On  January  21, 2011,  former  OFPP  Administrator  Daniel  Gordon  sent 
a  memorandum  to  Chief  Acquisition  Officers  and  Senior  Procurement 
Executives  on  enhancing  assessments  of  contractor  past  performance, 
which  included  steps  and  strategies  for  improving  the  collection  of 
information.  The  memorandum  emphasized  that  while  compliance  with 
reporting  requirements  is  important,  the  quality  of  the  reports  submitted 
is  far  more  crucial.  It  also  acknowledged  various  challenges  that  contrib¬ 
ute  to  the  low  number  and  quality  of  these  assessments,  which  include 
staff  shortages  and  the  transition  to  the  new  federal-wide  system  that 
integrates  the  PPIRS,  CPARS,  and  the  Federal  Awardee  Performance 
and  Integrity  Information  System  (Gordon,  2011,  p.  1). 
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Despite  efforts  by  the  DoD  and  other  government  organizations,  the 
policies  and  mandates  lack  enforcement  mechanisms  to  incentivize 
better  behavior  and  improve  past  performance  practices.  This  article 
recommends  solutions  that  will  strengthen  the  data,  processes,  and  tools 
associated  with  evaluating  past  performance,  and  provide  the  government 
with  a  strategy  to  benefit  from  the  implementation  of  these  resources. 

Problem 

The  DoD  evaluates  the  contractor’s  proposed  solution  vs. 

evaluating  the  contractor’s  record  of  actual  performance.  FAR 

Pt.  15.3  covers  the  selection  of  a  contractor  in  competitive  negotiations. 
According  to  the  FAR,  the  government  is  required  to  evaluate  three 
areas  for  acquisitions:  the  quality  of  the  proposed  product  or  service,  the 
price  or  cost  to  the  government,  and  past  performance  (for  acquisitions 
that  exceed  the  simplified  acquisition  threshold).  The  government  often 
requests  the  contractor  to  submit  volumes  covering  technical,  cost,  and 
past  performance  as  part  of  its  proposal.  The  technical  proposal  conveys 
the  contractor’s  proposed  technical  solution  or  response  to  a  require¬ 
ment;  the  cost  proposal  identifies  the  proposed  costs  of  delivering  the 
proposed  technical  solution;  and  the  past  performance  volume  shows 
how  the  contractor  performed  on  previous  similar  or  related  efforts  as 
an  early  indicator  of  potential  future  performance. 

Unfortunately,  the  DoD  currently  lacks  both  adequate  data  on  past 
performance  and  effective  processes  to  evaluate  a  company’s  qualifica¬ 
tions,  including  key  personnel.  Thus,  evaluation  of  past  performance 
is  not  used  as  effectively  as  it  could  be  to  predict  future  performance 
on  a  contract.  These  conclusions  are  borne  out  by  a  2009  study  by  the 
U.S.  Government  Accountability  Office  (GAO,  2009).  The  study  exam¬ 
ined  past  performance  practices  by  analyzing  62  procurements  in  five 
agencies  that  perform  major  acquisitions:  the  DoD,  the  Department  of 
Energy,  the  U.S.  General  Services  Administration  (GSA),  the  Department 
of  Homeland  Security,  and  the  National  Aeronautics  and  Space 
Administration.  The  study  found  that  82  percent  of  the  past  performance 
evaluations  did  not  contain  narratives  sufficiently  detailed  to  establish 
that  the  resulting  ratings  were  credible  or  justifiable  (GAO,  2009,  p.  8). 

In  many  of  these  acquisitions,  the  government  cited  a  company’s 
technical  approach  as  the  most  important  noncost  factor.  More  than 
60  percent  of  the  contracting  officers  stated  that  “past  performance  is 
rarely  or  never  a  deciding  factor  in  selecting  a  contractor”  (GAO,  2009, 
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p.  8).  However,  the  acquisition  requests  for  proposal  (RFP)  that  placed 
emphasis  on  past  performance  noted  that  this  encouraged  companies 
to  perform  better  (GAO,  2009,  p.  2).  The  report  succinctly  concluded: 
“Regardless  of  the  source  used,  contracting  officials  agreed  that  for  past 
performance  information  to  be  meaningful  in  contract  award  decisions, 
it  must  be  documented,  relevant,  and  reliable”  (GAO,  2009,  p.  8). 

Without  such  information,  the  government  evaluation  must  rely 
heavily  on  factors  (primarily  technical  and  cost)  in  the  contractor’s 
proposal  that  describe  a  hypothetical  situation:  howthe  contractor  plans 
to  perform  on  a  contract.  In  essence,  this  means  that  evaluations  are 
based  on  the  quality  of  a  document  produced  by  a  professional  proposal 
writing  team  during  the  45-  to  90-day  solicitation  timeframe.  These 
teams  specialize  in  showcasing  their  company’s  capabilities;  they  are 
not  necessarily  experts  in  devising  solutions  within  the  company’s 
technical  capabilities,  nor  in  describing  within  the  RFP  how  the  com¬ 
pany  can  execute  with  acceptable  risk.  Proposals  are  often  written  with 
little  or  no  input  from  the  staff  members  who  will  execute  the  day-to-day 
contract  requirements.  As  a  result,  proposals  submitted  for  competi¬ 
tive  source  selections  often  present  “optimistic”  solutions  that  carry 
significant  inherent  risk.  This  risk  often  materializes  during  contract 
execution,  which  directly  contributes  to  program  failures,  cost  overruns, 
and  schedule  delays,  which  are  endemic  to  the  DoD  acquisition  process. 
The  heavy  scrutiny  of  technical  and  cost  proposal  evaluations  can  prove 
to  be  a  wasted  exercise  when  contractors  continually  overrun  budgets 
or  require  major  engineering  change  proposals  throughout  the  life  of 
the  acquisition. 

Given  the  shortcomings  described  previously,  the  evaluation  of  past 
performance  often  becomes  a  superficial  pass/fail  test.  The  majority  of 
best  value  source  selections  require  the  government  to  evaluate  past  per¬ 
formance  on  the  basis  of  a  sliding  scale  that  ranges  from  high-confidence 
to  no-confidence.  The  DoD  uses  this  scale  to  assess  its  level  of  confidence 
in  a  contractor’s  ability  to  perform  based  on  previous  related  experience. 
Without  adequate  information  about  all  offerors,  the  DoD  is  reluctant  to 
place  strong  emphasis  on  past  performance,  especially  due  to  the  high 
risk  of  protest.  As  a  result,  the  DoD  usually  assigns  all  offerors  a  similar 
confidence  rating  to  level  the  playing  field,  and  uses  other  evaluation 
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factors  to  discriminate  among  proposals.  Therefore,  and  not  surpris¬ 
ingly,  the  government  often  fails  to  select  a  business  partner  with  an 
established  history  of  high  performance  and  relevant  experience. 

The  OFPP  conducted  a  pilot  test  with  30  contracts  that  used  past 
performance  as  a  significant  evaluation  factor.  Those  contracts  identi¬ 
fied  a  20  percent  increase  in  average  customer  satisfaction,  confirming 
the  assertion  that  the  increased  use  of  past  performance  data  supports 
positive  program  results  (GSA,  1997).  To  improve  how  the  DoD  lever¬ 
ages  past  performance  in  its  source  selections,  the  remainder  of  this 
section  reviews  proposed  solutions  that  incorporate  well-founded  best 
practices  to  mitigate  the  risks  associated  with  inadequate  past  perfor¬ 
mance  evaluations. 


Proposed  Solutions 

FIGURE.  PAST  PERFORMANCE  CYCLE 


The  Past  Performance  Cycle  depicted  in  the  figure  highlights  four 
stages  where  we  concentrated  our  research  to  improve  processes  and 
tools,  and  recommended  implementation  and  enforcement  of  policies. 

Alpha  contracting  negotiations  with  prequalified  vendor 

candidates.  Alpha  contracting  negotiations  with  prequalified  vendors 
leverage  the  best  practices  of  the  Qualifications-Based  Selection  (QBS) 
strategy  authorized  for  Architecture  and  Engineering  (A/E)  contracts 
under  FAR  Pt.  36.  The  QBS  strategy  was  originally  developed  because 
certain  creative  professional  services  cannot  be  fairly  priced  before  the 
creative  process  has  taken  place.  The  QBS  process  improves  innova¬ 
tion,  quality,  and  creativity  by  allowing  the  government  to  negotiate  the 
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contract  requirements  and  pricing  jointly  with  the  contractor  against 
an  independent  cost  estimate.  The  Brooks  Act  of  1972  delineates  the 
framework  for  QBS  using  the  following  four-step  phased  approach: 

1.  The  government  determines  the  desired  qualifications,  includ¬ 
ing  the  experience  and  demonstrated  competence  of  interested 
parties.  The  information  provided  by  the  offerors  should 
emphasize  technical  ability  (cost  is  not  considered  as  part  of 
the  initial  evaluation). 

2.  The  government  creates  a  short  list,  ranking  the  bidders  by 
their  qualifications. 

3.  The  government  conducts  interviews  with  the  firms  on  the  short 
list  and  then  reranks  the  firms. 

4.  Finally,  the  government  negotiates  a  statement  of  work  (SOW) 
and  fair  price  with  the  most  qualified  firm. 

Unfortunately,  the  FAR  only  allows  use  of  QBS  for  contracts  involv¬ 
ing  the  construction  or  building  trades,  including  transportation 
systems.  However,  the  DoD  could  leverage  some  of  the  benefits  offered 
by  the  A/E  procurement  strategy  by  combining  some  QBS  attributes 
with  other  contracting  best  practices  and  strategies.  For  example,  the 
DoD  can  use  a  prequalification  process  to  narrow  the  pool  of  qualified 
vendors,  similar  to  step  1  in  the  QBS  process.  This  can  be  done  by  using 
an  advisory  multistep  process  described  in  FAR  Pt.  15.202,  or  by  estab¬ 
lishing  a  competitive  range  based  entirely  on  past  performance  criteria. 
Narrowing  the  field  of  vendor  candidates  would  ensure  that  the  DoD 
optimizes  its  time  and  resources  by  conducting  RFP  activities  only  with 
the  vendors  that  have  the  highest  likelihood  of  being  selected  for  contract 
award.  This  approach  also  benefits  potential  contractors  since  they  do 
not  have  to  expend  valuable  proposal  development  funds  for  solicitations 
they  have  little  chance  of  winning. 

Processes  to  narrow  down  the  pool  of  qualified  vendors  on  the  basis 
of  past  performance  and  experience  can  require  up-front  time  and 
investment  from  the  government.  However,  these  strategies  can  save 
significant  time  during  the  “back-end”  of  the  proposal  evaluation  process. 
Ranking  the  vendors  also  helps  keep  competitive  pressures  in  play  during 


67 


Defense  ARJ,  April  3013,  Vol.  30 No.  1: 049-080 


Past  Performance  as  an  Indicator  of  Future  Performance: 
Selecting  an  Industry  Partner  to  Maximize  the  Probability  of  Program  Success 


negotiations.  The  government  mitigates  the  disadvantages  associated 
with  a  sole-source  negotiation  environment  because  it  maintains  the  flex¬ 
ibility  to  move  to  the  next-ranked  vendor  at  any  point  in  the  negotiations. 

Additionally,  the  DoD  can  use  a  question-and-answer  (Q&A)  session 
as  part  of  an  oral  presentation  strategy,  similar  to  the  QBS  interview 
process.  Open  communication  with  vendors  to  verify  experience  and 
qualifications  is  an  important,  but  often  overlooked  activity.  Usually 
the  government  accepts  the  qualifications  and  experience  stated  in  a 
company’s  proposal  at  face  value.  However,  greater  insight  regarding  the 
contractor’s  related  experience  and  the  qualifications  of  the  proposed  key 
personnel  would  increase  the  accuracy  of  the  government’s  assessment. 

Lastly,  the  DoD  can  apply  the  same  SOW  and  contract  price  nego¬ 
tiation  process  described  in  step  4  of  the  QBS  through  the  use  of  alpha 
contracting— an  innovative  strategy  that  allows  the  government  to 
perform  many  activities  jointly  with  the  contractor.  Together,  the  gov¬ 
ernment  and  contractor  develop  the  SOW  and  proposal  in  a  streamlined 
fashion.  This  process  offers  a  number  of  advantages  and  performance 
improvements,  such  as  enhancing  communication,  refining  and  clarify¬ 
ing  requirements,  and  ensuring  the  technical  solution  is  bounded  by  the 
capabilities  of  the  contractor.  The  alpha  contracting  process  also  has  the 
potential  to  yield  significant  savings  in  time  and  cost.  For  example,  the 
Army  PM-Tank  Main  Armament  Systems  M830A1  procurement  utilized 
the  alpha  contracting  process  and  reduced  the  lead-time  for  procure¬ 
ment  administration  by  55  percent,  thus  saving  $1  million,  which  was 
subsequently  used  to  buy  additional  units  (Jones,  2012,  p.  1). 

It  may  be  possible  to  combine  all  of  the  above  techniques  and  use  a 
GSA  Blanket  Purchase  Agreement  (BPA)  to  model  the  QBS  strategy  from 
steps  1  through  4.  Under  this  arrangement,  the  DoD  could  select  a  set  of 
contractors  from  a  GSA  schedule  and  use  a  prequalification  process  to 
narrow  the  field  of  vendors.  Next,  the  DoD  would  use  a  Q&A  process  to 
interview  candidates  and  rank  order  the  potential  vendors.  The  DoD  could 
then  utilize  the  alpha  contracting  process  to  negotiate  a  BPA  with  the 
highest  ranked  firm.  If  negotiations  with  the  top-ranked  vendor  resulted 
in  unfair  pricing,  or  if  the  DoD  and  the  vendor  were  unable  to  agree  on  a 
SOW,  the  DoD  would  have  the  right  to  conduct  negotiations  with  the  sec¬ 
ond-ranked  firm.  When  negotiations  are  complete,  the  DoD  would  issue 
a  BPA  using  a  GSA  schedule  contract  authorized  under  FAR  Pt.  8.405-3. 
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This  GSA  BPA  approach  has  certain  restrictions  that  may  limit 
applicability  across  a  broad  range  of  acquisitions  (e.g.,  GSA  BPAs  are 
limited  to  commercial  products  and  services  and  do  not  allow  cost-type 
contracts)  (GAO,  2009,  p.  1).  Furthermore,  the  QBS  approach  is  cur¬ 
rently  limited  to  A/E  contracts  authorized  under  FAR  Pt.  36.  However, 
the  need  for  creativity  and  innovation  is  not  limited  to  A/E  procure¬ 
ments.  Agile  software  development,  for  example,  values  flexibility, 
innovation,  and  collaboration,  and  can  greatly  benefit  from  a  QBS-like 
acquisition  approach. 

As  a  result,  the  DoD  should  pursue  an  amendment  to  FAR  Pt.  15  to 
allow  a  QBS-like  process  that  can  be  leveraged  for  many  different  types  of 
acquisitions.  The  DoD  can  achieve  maximum  benefit  from  this  approach 
if  the  amendment  allows  a  prequalification  process  to  narrow  the  field  of 
potential  vendors.  An  interview  or  Q&A  process  can  facilitate  the  vendor 
ranking  process,  and  alpha  contracting  would  allow  the  DoD  to  negotiate 
a  SOW  and  pricing  based  on  the  ranking  of  firms.  All  of  these  individual 
strategies  are  permitted  under  different  areas  of  the  FAR;  however,  a  FAR 
Pt.  15  amendment  would  allow  the  DoD  to  combine  them  into  a  single 
process  to  obtain  maximum  benefit. 

Solution  demonstrations  as  a  source  selection  technique. 

The  limited  availability  of  past  performance  information  makes  it  dif¬ 
ficult  to  verify  the  relevance  of  previous  experience.  As  an  alternative 
strategy,  the  DoD  can  use  demonstrations  as  part  of  the  source  selec¬ 
tion  process  to  validate  previous  experience.  A  demonstration  could 
include  any  material  representation  of  past  experience  (e.g.,  prototypes, 
software  demonstrations).  Using  this  strategy,  the  DoD  would  require 
vendors  to  submit  a  functioning  prototype  or  a  previously  developed 
relevant  product,  and  to  demonstrate  its  capabilities  as  part  of  the  pro¬ 
posal  process.  The  DoD  could  apply  this  strategy  to  both  hardware  and 
software  procurements;  for  example,  screen  mock-ups  and  software 
demonstrations  from  previous  efforts  can  be  used  to  evaluate  certain 
software  procurements.  The  demonstration  could  be  submitted  prior 
to  the  formal  proposal  process,  or  could  be  used  to  supplement  a  written 
technical  proposal.  The  proposal  instructions  would  specify  the  desired 
functionality  of  the  demonstrated  product,  and  thoroughly  describe  how 
the  DoD  would  evaluate  the  demonstration. 
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This  source  selection  strategy  will  not  only  help  to  narrow  the  pool  of 
qualified  vendors,  but  will  also  offer  a  way  for  vendors  to  establish  recent 
and  relevant  experience.  The  demonstration  becomes  a  tangible  piece  of 
evidence  that  the  contractor  has  verified  and  proven  its  experience  with 
the  technology  and/or  solution  proposed. 


Problem 

The  DoD  lacks  consistent  and  thorough  processes  to  evaluate 

key  personnel.  The  overall  experience  and  business  processes  of  the 
contractor  are  central  to  evaluation  of  past  performance.  However,  the 
qualifications  of  the  personnel  proposed  for  key  roles  on  the  contract  have 
even  greater  importance  for  the  future  success  of  the  program.  The  capa¬ 
bilities  and  attributes  of  the  key  personnel  proposed  should  be  directly 
relevant  to  the  successful  completion  of  the  program’s  requirements 
and  supported  by  the  strength  of  the  company’s  experience.  The  DoD 
processes  should  aspire  not  only  to  select  the  strongest  contractor,  but 
also  to  ensure  that  the  contractor  assigns  the  “A  Team”  to  the  program. 
When  companies  know  their  key  personnel  will  be  closely  evaluated,  they 
have  an  incentive  to  offer  their  best  performers,  rather  than  simply  per¬ 
sonnel  who  meet  minimal  qualification  requirements.  Top-performing 
personnel  with  a  proven  track  record  will  also  be  inclined  to  ensure 
the  technical  solution  proposed  can  be  executed  with  acceptable  risk, 
potentially  providing  internal  checks  and  balances  with  the  proposal 
writing  team. 
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Today,  contractors  generally  identify  key  personnel,  and  include 
their  resumes  as  part  of  the  proposal  package,  but  most  source  selections 
limit  evaluation  of  these  key  personnel  to  a  simple  check  of  whether  the 
proposed  team  members  meet  experience  criteria.  The  DoD  should  have 
consistent  and  thorough  processes  for  the  evaluation  of  key  personnel 
that  go  beyond  a  checklist  of  experience  criteria.  This  article  recom¬ 
mends  that  the  DoD  increase  the  use  of  oral  presentations  and  Q&A 
sessions  to  verify  the  qualifications  of  key  personnel. 

Proposed  Solution 

Evaluate  key  personnel  using  oral  presentations.  Throughout 
the  government,  oral  presentations  are  often  used  in  conjunction  with 
written  proposals  to  clarify  or  support  all  or  some  aspects  of  the  contrac¬ 
tor’s  technical  and/or  management  proposal  (Sade,  2009,  p.  1).  The  U.  S. 
Air  Force  Source  Selection  Procedures  Guide  states  that  discussions  may 
be  either  oral  or  written,  and  specifically  requires  such  discussions  for 
those  areas  of  the  proposal,  such  as  past  performance,  that  are  signifi¬ 
cant  enough  to  affect  the  source  selection  decision  (U.S.  Air  Force,  2000). 
These  presentations  can  greatly  enhance  the  evaluation  process.  They 
give  the  government  an  opportunity  to  evaluate  the  effectiveness  of  the 
contractor’s  communication  style,  clarify  the  contractor’s  proposal,  and 
assess  important  attributes  and  group  dynamics  of  the  contractor’s  key 
personnel  team.  In  addition,  oral  presentations  remove  the  gloss  created 
by  professional  proposal  writers  and  give  the  government  an  opportunity 
to  interact  directly  with  the  contractor  team  that  will  perform  the  day-to- 
day  tasks  required  in  the  contract. 

The  DoD  should  take  advantage  of  this  opportunity  to  interact 
directly  with  key  vendor  personnel.  To  further  enhance  the  benefits  of 
these  interactions,  the  DoD  should  mandate  that  proposed  key  person¬ 
nel  present  the  oral  proposals.  This  prevents  contractors  from  using 
their  best  marketing  representatives  to  make  presentations,  and  gives 
the  government  an  opportunity  to  evaluate  the  overall  strength  of  the 
contractor  team  that  will  perform  the  actual  program  tasks. 

In  addition,  the  DoD  should  use  the  Q&A  portion  of  the  oral  presenta¬ 
tion  to  verify  the  qualifications  and  experience  of  key  personnel.  The  Air 
Force  recommends  maximum  use  of  oral  presentations  to  complement 
or  replace  written  technical  volumes  where  they  best  fit  the  acquisition 
(U.S.  Air  Force,  2000,  p.  10).  The  DoD  can  review  the  resumes  of  the 
proposed  personnel  ahead  of  time,  and  prepare  targeted  questions  that 
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will  help  the  government  understand  the  breadth  and  depth  of  the  team 
member’s  actual  experience  relative  to  the  program’s  requirements. 
The  government  should  structure  questions  according  to  behavioral 
interviewing  techniques:  asking  questions  that  require  the  candidates 
to  describe  a  past  situation  in  which  they  exhibited  a  specific  capability, 
such  as  problem  solving,  teamwork,  or  planning  and  organizing. 

The  premise  behind  behavioral  interviewing  is  that  the  most  accu¬ 
rate  predictor  of  future  performance  is  past  performance  in  similar 
situations.  This  strategy  is  used  by  more  than  70  percent  of  Fortune  500 
companies,  which  credit  it  with  being  55  percent  predictive  of  future 
on-the-job-behavior,  while  traditional  interviewing  has  been  deter¬ 
mined  to  be  only  10  percent  predictive  (Hansen,  n.d.,  p.  1).  This  method 
of  interviewing  provides  two  types  of  information  that  improve  insight 
regarding  proposed  key  personnel:  (a)  examples  of  how  they  actually 
exhibited  the  desired  behavior,  versus  a  hypothetical  discussion,  and  (b) 
evidence  of  the  success  or  failure  of  the  solution. 

The  DoD’s  evaluation  of  key  personnel  should  go  beyond  verifying  a 
checklist  of  experience  criteria.  Key  considerations  must  include  formal 
assessments  of  the  contractor’s  trustworthiness  with  respect  to  foreign 
influence  and  control,  the  rigor  of  corporate  security  protocols,  and  the 
integrity  of  the  contractor’s  global  supply  chain  relationships. 

Lastly,  the  contract  should  include  key  personnel  clauses  that  pro¬ 
tect  the  government  from  “bait  and  switch”  tactics.  In  some  instances, 
contractors  have  replaced  the  team  originally  proposed  with  potentially 
less  qualified  candidates  shortly  after  contract  award.  To  mitigate  this 
risk,  the  DoD  contracts  should  require  that  the  key  personnel  originally 
proposed  on  the  contract  remain  with  the  program  for  a  minimum  time 
period  (e.g.,  12-18  months),  unless  that  individual  leaves  the  company. 
This  becomes  especially  important  if  the  government  uses  key  personnel 
as  an  important  evaluation  factor  in  the  source  selection  process. 

Problem 

The  DoD  lacks  the  adequate  tools  and  information  to  collect, 
analyze,  and  report  past  performance  information  consistently 
acrosscontracts  A  high-  quality  data  repository  greatly  strengthens  the 
government’s  ability  to  evaluate  past  performance  as  part  of  the  source 
selection  process.  Well-documented  information,  easily  accessible  in 
such  repositories,  enables  the  government  to  defend  its  position  during  a 
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contract  protest.  Thus,  a  data-driven  process  to  collect,  analyze,  and  report 
past  performance  information  is  critical  in  allowing  the  government  to 
incorporate  past  performance  in  a  source  selection  decision. 

Two  important  past  performance  tools  used  today  are  CPARS  and 
PPIRS.  CPARS  is  a  suite  of  Web-enabled  applications  that  document 
contractors’  past  performance  in  accordance  with  the  FAR.  It  uses 
electronic  workflow  to  automate  contracting  officers’  evaluations  of 
contractor  performance.  It  also  allows  contractors  to  submit  comments 
electronically  in  response  to  a  government  assessment  and  either  concur 
or  nonconcur.  The  DoD -developed  PPIRS  is  a  Web-enabled  government 
enterprise  application  supporting  source  selections.  Since  the  FAR 
requires  all  federal  agencies  to  post  contractor  performance  evaluations 
in  PPIRS,  it  now  acts  as  the  government- wide  warehouse  for  information 
on  contractor  past  performance  (PPIRS,  n.d.,  p.  1).  However,  it  should  be 
noted  that  while  all  federal  acquisitions  are  required  to  use  the  PPIRS, 
not  all  government  organizations  have  complied.  As  a  warehouse,  it 
allows  query-based,  read-only  retrieval  and  review  of  contractor  data 
by  qualified  government  acquisition  personnel.  Contractors  can  view 
their  own  data  by  using  a  central  contractor  registration  to  gain  access. 

Although  the  FAR  requires  the  government  to  evaluate  past  per¬ 
formance,  the  collection  and  reporting  of  past  performance  data  have 
been  inconsistent  and  often  untimely.  The  lack  of  consistent,  available, 
and  reliable  data  hinders  the  DoD’s  ability  to  make  past  performance  an 
effective  part  of  the  proposal  evaluation  process.  The  government  has 
recently  made  improvements  to  many  of  the  past  performance  tools, 
including  CPARS  and  PPIRS,  to  provide  commonality  of  data  entry 
format  and  increase  the  availability  of  data  for  retrieval.  Despite  these 
improvements,  current  data  repositories  still  perform  their  functions 
inadequately,  or  are  used  inaccurately  across  the  DoD.  The  problems 
with  the  government’s  current  data  repositories  have  plagued  the  DoD  for 
years.  Poor  data  quality,  incompatible  data  formats,  inconsistent  report¬ 
ing  timeliness,  lack  of  data  entry,  diffused  accountability,  and  subjective 
evaluations  are  among  the  issues  associated  with  the  way  the  DoD  has 
collected,  analyzed,  and  reported  past  performance  information. 
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Proposed  Solutions 

Implement  and  enforce  the  DoD  policies  for  past  performance. 
The  OFPP  and  the  Director,  Defense  Procurement  and  Acquisition 
Policy  (DPAP),  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  (OUSD[AT&L])  have  recently  issued  guid¬ 
ance  and  policy  that  emphasize  the  importance  of  information  on  past 
performance.  Current  policies,  however,  lack  appropriate  enforcement 
mechanisms  to  make  this  policy  effective  and  actionable.  The  Director, 
DPAP  and  OUSD(AT&L)  should  therefore  issue  a  DoD -wide  mandate  to 
populate  past  performance  databases  in  a  timely  and  consistent  man¬ 
ner.  Per  FAR  Pt.  42.15,  the  responsibility  for  reporting  past  performance 
resides  with  the  contracting  officer,  but  it  is  important  to  make  this  a 
shared  responsibility  between  the  PMO  and  the  contracting  office.  The 
program  team  (e.g.,  program  manager,  contracting  officer’s  technical 
representative)  should  evaluate  how  well  the  contractor  is  performing 
against  the  SOW  requirements.  The  contracting  office  should  collect 
this  information  from  the  PMO,  and  also  provide  input  on  the  contrac¬ 
tor’s  performance  against  the  contract  requirements  (e.g.,  reporting 
requirements,  terms,  and  conditions).  The  contracting  office  should  have 
ultimate  responsibility  for  collecting  this  information  and  populating  the 
past  performance  database  in  a  timely  and  consistent  manner. 

The  DoD  should  increase  enforcement  of  this  policy  through  a  vari¬ 
ety  of  mechanisms.  Agencies  should  be  required  to  report  metrics  on 
how  well  they  are  complying  with  this  mandate.  The  Director,  DPAP 
and  OUSD(AT&L)  should  collect  these  metrics  on  a  monthly  basis  and 
publish  the  results  on  DPAP’s  Web  site.  The  head  of  each  Combatant 
Command,  Service,  or  Agency  (C/S/A)  should  be  held  accountable  for 
C/S/A-wide  compliance  with  this  policy. 

Additionally,  the  DoD  should  require  that  past  performance  reviews 
become  a  mandatory  part  of  acquisition  program  reviews  by  the  Program 
Executive  Office.  This  will  ensure  that  program  offices  are  held  account¬ 
able  for  meeting  this  policy  requirement.  As  a  further  incentive,  the  DoD 
should  prohibit  programs  from  exercising  option  awards,  award  terms, 
or  award  fee  payouts  until  they  have  populated  past  performance  infor¬ 
mation  into  a  database. 
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Lastly,  the  DoD  should  include  an  evaluation  of  compliance  with  this 
process  as  part  of  each  contracting  officer’s  annual  pay  and  performance 
review.  Metrics  that  report  compliance  with  this  mandate  at  the  indi¬ 
vidual  contracting  officer  level  will  further  enhance  the  enforcement  of 
this  policy. 

Identify  quantifiable  metrics  for  agency-wide  reporting. 

Past  performance  evaluations  today  tend  to  be  based  on  subjective  judg¬ 
ments  by  the  evaluator.  This  makes  it  difficult  to  compare  performance 
across  programs  and  contractors.  Additionally,  observations  show 
that  program  managers  and  contracting  officers  are  often  reluctant  to 
report  negatively  on  past  performance  because  this  can  reflect  poorly 
on  their  own  ability  to  manage  the  program  or  contract.  Furthermore, 
to  avoid  conflict  with  the  contractor,  the  government  may  refrain  from 
documenting  performance  deficiencies  in  official  databases.  As  a  result, 
the  past  performance  write-up  does  not  always  reflect  a  contractor’s 
performance  accurately. 

The  DoD  should  agree  on  a  set  of  quantifiable  metrics  to  capture 
objective  data  within  standardized  past  performance  databases.  The  use 
of  such  quantifiable  and  simple  yes/no  metrics  can  provide  a  consistent 
and  reliable  way  to  compare  contractor  performance  across  contracts. 
For  example,  Earned  Value  Management  (EVM)  data  can  be  used  to 
report  cost  and  schedule  performance  on  complex  acquisitions.  Award 
fee  determinations  can  also  provide  a  metric,  since  contractors  often  use 
this  metric  to  measure  their  own  performance.  Simple  yes/no  metrics 
can  be  used  for  non-EVM  contracts.  Such  questions  could  include:  Did 
the  contractor  perform  within  proposed  cost?  Did  the  contractor  deliver 
on  time?  Did  the  program  encounter  a  Nunn-McCurdy  breach?  Each  of 
the  metrics  and  questions  can  be  followed  by  a  data  field  that  allows  the 
government  to  explain  the  evaluations  provided  in  greater  detail. 

Include  validated  and  approved  contract  deliverables  in  past 
performance  databases.  The  DoD  can  increase  the  quantity  and 
quality  of  the  data  in  past  performance  databases  by  incorporating  infor¬ 
mation  on  validated  and  approved  contract  deliverables.  For  example, 
summary  narratives  on  award  fees  provide  a  wealth  of  information  on 
how  the  contractor  performed  during  the  award  fee  period.  This  type  of 
information  can  be  very  useful  during  evaluations  of  past  performance, 
since  it  gives  the  DoD  a  more  complete  picture  of  how  well  the  contractor 
performed  over  the  life  of  a  contract. 
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Lastly,  past  performance  databases  should  include  a  mechanism  for 
uploading  documents  directly  into  CPARS  to  share  with  other  govern¬ 
ment  counterparts.  This  would  greatly  enrich  the  quality  and  quantity 
of  past  performance  data. 

Conclusions  and  Recommendations 

The  lack  of  adequate  past  performance  data,  tools,  and  processes 
hinders  the  government  from  effectively  evaluating  the  qualifications 
of  companies  and  key  personnel.  The  DoD  tends  to  deemphasize  past 
performance  evaluations  during  the  competitive  source  selection  pro¬ 
cess,  largely  because  data  that  would  allow  it  to  discriminate  among 
proposals  are  unavailable.  Current  government  tools  have  proven  inad¬ 
equate  to  collect,  analyze,  and  report  information  on  past  performance. 
The  absence  of  timely,  reliable,  documented  data  impedes  the  entire 
cycle  of  collecting,  reporting,  and  evaluating  past  performance  during 
source  selection  (see  figure).  The  inability  to  use  past  performance  as  a 
predictor  of  future  contract  success  places  the  DoD  at  risk  of  repeating 
program  mistakes. 

Evaluating  a  company’s  performance  record  allows  the  government 
to  assess  the  effectiveness  of  a  contractor’s  management  processes,  tools, 
and  resources.  Because  effective  performance  depends  on  qualified  and 
experienced  personnel  who  know  how  to  replicate  success,  the  ability 
to  evaluate  proposed  key  personnel  also  becomes  an  important  aspect 
of  estimating  a  company’s  ability  to  meet  the  contract  requirements. 
Yet,  the  DoD  is  not  utilizing  the  necessary  processes  to  fully  assess  the 
capabilities  of  the  key  personnel  proposed  on  a  contract. 

The  DoD  currently  lacks  a  comprehensive  approach  to  tackle  the 
issues  identified  in  this  article.  Since  the  problems  are  interdependent, 
they  require  a  holistic  approach  that  addresses  the  tools,  processes,  and 
policies  surrounding  the  past  performance  cycle  pictured  in  the  figure. 
The  processes  will  not  improve  without  adequate  tools  to  provide  the 
data.  They  will  not  be  effective  without  the  proper  policies  in  place  to 
enforce  them,  and  they  cannot  be  implemented  without  the  proper  pro¬ 
cesses  and  tools  to  support  them. 
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The  recommendations  summarized  below  have  been  brought  for¬ 
ward  throughout  this  article  as  important  steps  toward  improving  the 
DoD’s  probability  of  achieving  successful  future  program  outcomes: 

1.  Use  alpha  contracting  negotiations  with  prequalified  vendor 
candidates.  Amend  FAR  Pt.  15  to  allow  for  a  QBS-like  approach 
across  a  broader  range  of  acquisitions. 

2.  Use  solution  demonstrations  as  part  of  source  selection  to  increase 
accuracy  in  evaluations  of  the  contractor’s  performance  record. 

3.  Include  oral  presentations  in  the  evaluation  process;  improve 
the  consistency  and  depth  of  personnel  evaluations  by  using  the 
behavioral  Q& A  process  to  verify  the  relevant  qualifications  and 
experience  cited  in  proposed  key  personnel  resumes. 

4.  Issue  a  policy-wide  mandate  that  enforces  past  performance  data 
entry  standards.  Use  an  online  tool  to  track  and  report  compli¬ 
ance  with  this  policy  requirement. 

5.  Agree  on  a  set  of  quantifiable  metrics  to  capture  objective  data 
within  standardized  past  performance  databases. 

6.  Increase  the  quantity  and  quality  of  data  in  past  performance 
databases  by  incorporating  relevant  contract  performance  docu¬ 
mentation  from  validated  and  approved  contract  deliverables. 
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Acquisition  Program 
Funding  Stability— A  Myth 


(  COL  Robert  D.  Morig,  USA  (Ret.) 


Program  stability  and  funding  stability  are  continu¬ 
ously  promoted  as  key  to  successful  acquisition  reform. 
Funding  stability,  according  to  prevailing  wisdom, 
leads  to  program  stability.  Unfortunately,  the  dynamic, 
evolving,  and  methodical  requirements  generation, 
technology  enhancement,  and  resourcing  processes 
prevalent  throughout  the  Department  of  Defense 
(DoD)  are  not  conducive  to  funding  stability.  This  article 
discusses  results  from  a  survey  of  financial  management 
practitioners  that  provide  insight  into  factors  that  both 
enable  and  detract  from  achieving  funding  stability.  The 
author  presents  program  stability  as  a  myth  in  the  real 
world  environment  where  the  “norm”  is  characterized 
by  changing  program  requirements,  technologies,  and 
funding.  He  further  hypothesizes  that  stability  cannot 
occur  without  major  change  in  the  Planning,  Program¬ 
ming,  Budgeting  and  Execution,  and  Congressional 
Enactment  processes. 
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Program  funding  has  always  been  the  Achilles  heel  of  acquisition 
programs  during  their  development  and  production.  There  never  seems 
to  be  enough  precious  funds  available.  On  May  8, 2010,  at  the  Eisenhower 
Library  in  Abilene,  Kansas,  then  Secretary  of  Defense  Robert  M.  Gates 
gave  a  speech  centered  on  lowering  program  costs  (Gates,  2010a).  He  said 
that  he  would  be  directing  the  military  services,  the  joint  staff,  and  others 
to  examine  how  the  DoD  could  reduce  overhead  costs  and  transfer  those 
savings  into  force  structure  and  weapon  systems  modernization  gains. 


Acquisition  Leadership- 
Making  the  Hard  Choices 

Shortly  thereafter,  then  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  Ashton  B.  Carter  generated  a  memorandum 
entitled  “Better  Buying  Power:  Mandate  for  Restoring  Affordability  and 
Productivity  in  Defense  Spending”  (Carter,  2010a).  He  provided  more 
specific  direction  to  acquisition  professionals  and  emphasized  the  need 
to  restore  affordability  to  our  programs  and  activities.  On  August  16, 
2010,  Gates  sent  a  memorandum  to  the  key  department  leaders  entitled 
“Department  of  Defense  (DoD)  Efficiency  Initiatives,”  directing  a  series 
of  20  initiatives  to  reduce  duplication,  overhead  and  excess,  and  instill  a 
culture  of  savings  and  restraint  across  the  DoD  (Gates,  2010b). 
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On  August  20,  Gates  followed  up  with  another  memorandum  to 
key  DoD  leaders  entitled,  “Guidance  on  DoD  Efficiency  Initiatives  with 
Immediate  Application,”  directing  five  immediate  actions  to  identify  effi¬ 
ciencies  and  resultant  savings  (Gates,  2010c).  Shortly  thereafter.  Carter 
released  another  memorandum  for  acquisition  professionals  (Carter, 
2010b).  On  September  14, 2010,  he  outlined  23  specific  actions  organized 
into  five  broad-major  areas  and  noted  there  would  be  continued  budget 
turbulence,  if  the  acquisition  community  chose  not  to  pursue  greater 
efficiencies.  One  of  those  actions  was  to  “make  production  rates  economi¬ 
cal  and  hold  them  stable”  (p.  4),  implying  some  sort  of  funding  stability 
to  enable  this  initiative.  Other  funding  stability-related  actions  in  the 
memorandum  were  to  “mandate  affordability  as  a  requirement”  (p.  2), 
“set  shorter  program  timelines  and  manage  to  them”  (p.  4),  “address 
schedule  directly  as  an  independent  variable”  (p.  5),  and  “Increase  the 
use  of  Fixed-Price  Incentive  Firm  Target  (FPIF)  contract  type  where 
appropriate  using  a  50/50  share  line  and  120  percent  ceiling  as  a  point 
of  departure”  (p.  6).  To  meet  these  objectives  requires  stable  programs 
and  stable  funding. 

To  emphasize  his  support,  Air  Force  Secretary  Michael  Donley,  in  a 
speech  at  the  Air  Force  Association  Conference  on  September  13, 2010, 
provided  guidance  to  Air  Force  Leaders  and  reminded  them  “not  to  get 
‘over-extended’”  with  more  programs  and  resource  commitments  than 
we  can  afford  (Kreisher,  2010,  para.  7).  He  encouraged  them  to  seek 
sufficient  funding  to  ensure  success  without  leaving  programs  broken, 
underfunded,  disconnected  in  the  next  budget  cycle,  and  a  bill  payer  for 
other  programs.  Donley  summed  up  the  guidance  with  “make  the  hard 
choices  now”  (para.  8). 

General  Ray  Odierno,  in  July  2011  Senate  confirmation  hearings 
to  be  the  next  Army  Chief  of  Staff,  stated  “carefully  refined  contract 
requirements,  a  sound  program  strategy,  and  stable  funding,”  are  nec¬ 
essary  to  get  the  procurement  situation  under  control  (Odierno,  2011). 

And  in  Defense  Acquisition  University  (DAU)  President  Katrina 
McFarland’s  Senate  confirmation  hearing  to  be  the  Assistant 
Secretary  of  Defense  (Acquisition)  on  March  12,  2012,  she  responded  to 
advance  questions  on  Funding  and  Requirements  Stability  by  stating, 
“Implementation  of  Affordability  Targets  at  Milestone  A,  Affordability 
Requirements  at  Milestone  B,  and  working  to  build  realistic  schedules 
and  hold  programs  to  them  are  recommended  steps.  Combined  with 


Defense  ARJ,  April  3013,  Vol.  SO  No.  1: 071-098 


84 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


the  Configuration  Steering  Board  process,  these  steps  as  described  in 
the  Better  Buying  Power  (BBP)  will  increase  the  program  funding  and 
requirements  stability”  (McFarland,  2012,  p.  20)  . 

Interestingly  enough,  long  before  these  most  recent  initiatives, 
congressional,  administration,  and  industry  leaders  already  declared 
program  stability  and  funding  stability  a  prerequisite  for  acquisition 
reform.  In  testimony  before  the  House  Armed  Services  Subcommittee  on 
Defense  Acquisition  Reform  in  September  2009,  Richard  Sylvester,  vice 
president,  Acquisition  Policy,  Aerospace  Industries  Association ,  recom¬ 
mended  that  the  government  move  to  stabilize  program  requirements, 
budgets,  and  system  configuration  (The  Department  of  Defense,  2009). 

On  September  1,  2009,  OSD  Director,  Acquisition  Resources  and 
Analysis  Nancy  Spruill  addressed  questions  on  cost  growth  stating, 
“We  have  found  that  funding  and  requirements  stability  and  greater 
technology  maturity  drive  successful  programs”  (Spruill,  2009,  p.  6). 
Later  in  that  same  article,  when  discussing  strategic  acquisition  reform, 
she  again  stated  the  need  to  establish  a  fixed,  stable  investment  budget. 
Findings  from  hundreds  of  acquisition  reform  initiatives  over  the  years 
reflect  the  same  conclusions. 

So  why  is  this  so  hard?  For  years,  the  acquisition  community  has 
been  saying  program  and  funding  stability  are  essential  to  acquisition 
reform.  To  that  end,  the  Department  of  Defense  leadership  has  con¬ 
sistently  directed  and  encouraged  acquisition  practitioners  to  ensure 
program  and  funding  stability.  And  yet,  after  all  the  repeated  emphasis, 
the  acquisition  community  has  not  been  able  to  meet  this  goal. 

Survey  Results— Detractors  &  Enablers 

A  survey  of  Financial  Management  professionals,  which  represents 
the  basis  of  this  article,  offers  some  insight  into  why  practitioners  in  the 
field  believe  funding  stability  is  so  problematic.  The  DAU  survey  was 
conducted  by  requesting  the  graduates  of  DAU  resident  courses  BCF- 
205,  BCF-211,  and  BCF-301  identify  three  detractors  and  three  enablers 
to  funding  stability.  The  survey  was  sent  to  the  graduates  of  20  classes 
(approximately  400  Financial  Management  workforce  personnel  from 
all  Services,  from  installations  across  the  continental  United  States  and 
outside  the  continental  United  States;  all  grades  from  GS-9  to  GS-15, 
including  program  management  offices  and  nonprogram  management 
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offices).  Forty  students  provided  responses  that  were  accumulated  into 
the  survey  results.  The  author  views  this  population  as  representative 
of  the  Financial  Management  community. 

Table  1  shows  the  results  from  a  DAU  survey  identifying  detractors 
to  stable  funding.  The  top  three  and  nine  of  16  detractors  identified  by  the 
survey  results  are  actually  caused  by  external  agencies  at  a  level  above 
the  program  office’s  control. 

TABLE  1.  FUNDING  STABILITY  DETRACTORS 


Top  Detractor— Continuing  Resolution 

The  top  detractor,  Continuing  Resolution,  occurs  when  Congress  is 
unable  to  complete  passage  of  Defense  Appropriations  bills  by  the  end 
of  the  fiscal  year.  Congress  routinely  passes  a  continuing  resolution  to 
fund  the  government  at  some  partial  level  based  upon  expenditure  rates 
of  the  previous  year  or  most  recent  congressional  marks.  In  Fiscal  Year 
(FY)  2011,  the  Federal  Government  was  forced  to  operate  under  seven 
continuing  resolutions  (the  first  from  October  1  to  December  3,  and  the 
last  expiring  April  15, 2011).  Continuing  resolutions  permit  the  govern¬ 
ment  to  continue  to  operate,  but  obviously  at  some  constrained  level 
of  funding.  New  start  programs  are  not  permitted  under  a  continuing 
resolution.  On  numerous  occasions.  Congress  had  to  pass  more  than  one 
continuing  resolution  before  reaching  an  agreement  on  an  appropriations 
bill  or  an  all-encompassing  omnibus  funding  bill.  The  resulting  incre¬ 
mental  funding  of  programs  has  caused  significant  funding  instability 
for  the  acquisition  community.  In  18  of  the  last  20  years  and  30  of  the 
last  33  years,  Congress  had  not  passed  a  Defense  Appropriations  bill  by 
October  1  and  had  to  resort  to  a  CRA,  or  Continuing  Resolution  Authority 
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(Streeter,  2011).  The  latest  is  the  6-month  FY13  CRA  funding  programs 
at  F  Y 12  levels.  As  a  financial  necessity,  operating  under  a  CRA  requires 
adjusting  to  this  incremental  flow  of  funding  to  projects.  Obviously,  this 
external  agency  (Congress)  has  the  capacity  to  significantly  impact  a 
program  office’s  ability  to  maintain  any  semblance  of  funding  stability. 
Based  on  past  history,  continuing  resolutions  will  almost  certainly  con¬ 
tinue  into  future  years,  resulting  in  further  program  funding  instability. 

Second  Top  Detractor— Headquarters  Changes 

The  second  detractor,  Headquarters  Changes,  is  related  to  changes 
from  higher  headquarters,  which  are  identified  in  the  survey  as  having 
the  most  impact  on  program  stability.  These  changes  also  make  funding 
stability  extremely  problematic.  Technology  advancements  constitute  a 
significant  portion  of  these  changes.  Technology  is  forever  emerging  to 
provide  the  warfighter  with  enhanced  capability.  Would  we  really  want 
to  ignore  a  technology  that  provides  a  significant  increase  in  warfighting 
capability?  In  today’s  Information  Technology  (IT)  systems,  software 
enhancements  in  speed,  storage  capacity,  or  throughput  volume  undoubt¬ 
edly  provide  greater  capabilities  to  existing  programs.  Do  we  really  want 
to  minimize  these  changes  and  field  obsolete  technologies  to  obtain 
program  and  funding  stability? 

Improved  optics,  greater  efficiencies  in  fuel  consumption,  lighter 
and  stronger  materials,  and  power  generation  technologies  are  all  exam¬ 
ples  of  technology  advancements  that  could  have  dramatic  impact  on 
warfighting  capability,  and  could  be  solutions  to  previously  identi¬ 
fied  capability  gaps.  Perhaps,  technology  advancements  in  logistics 
could  help  reduce  mortality  rates  of  our  soldiers  and  Marines  mov¬ 
ing  Petroleum,  Oil,  and  Lubricant  convoys  in  Afghanistan.  The  Army 
recently  completed  a  study  showing  .42  casualties  per  fuel  convoy  and  .34 
casualties  per  water  convoy  in  Afghanistan  (Army  Environmental  Policy 
Institute,  2009).  Understandably,  senior  leaders  at  a  higher  headquarters 
would  encourage  and  support  requirements  changes  driven  by  technol¬ 
ogy  improvements  that  save  lives  and  improve  capabilities.  Inevitably, 
requirements  and  technology  enhancements  will  continue  to  inhibit  the 
likelihood  of  program  stability  and  any  concomitant  funding  stability. 
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Third  Top  Detractor— Changes  In  Requirements 

The  third  most  important  detractor,  Changes  in  Requirements,  can 
be  more  easily  managed.  However,  eliminating  or  reducing  require¬ 
ments  changes,  or  the  impact  of  such  changes,  has  always  been  a 
challenge  for  the  acquisition  community,  charged  with  finding  a  mate¬ 
rial  solution  to  an  existing  system  that  no  longer  meets  the  warfighter’s 
requirements.  An  extensive  infrastructure  exists  to  analyze  existing 
warfighting  capabilities  against  specific  capability  gaps  identified  by 
combatant  commanders.  These  gaps  sometimes  take  on  a  life  all  their 
own.  As  an  illustration,  if  the  capability  exists  to  observe  emplacement 
of  Improvised  Explosive  Devices  (IED)  out  to  a  maximum  range  of  6 
kilometers,  then  the  warfighter  would  ideally  like  to  see  beyond  6  kilo¬ 
meters.  If  an  unmanned  aerial  drone  can  loiter  for  10  hours,  then  the 
warfighter  would  ideally  like  the  drone  to  loiter  for  more  than  10  hours. 
The  warfighter  would  surely  want  a  troop  carrier  that  provides  greater 
protection  at  less  weight,  or  a  radio  battery  that  lasts  twice  as  long.  And 
the  list  goes  on.  Seldom  will  Service  leadership  suppress  a  requirement 
that  would  give  the  warfighter  this  enhanced  capability.  Invariably, 
requirements  changes  will  continue  unabated.  Consequently,  funding 
instability  related  to  these  requirements  changes  will  continue  as  long 
as  this  country  has  soldiers,  sailors,  Marines,  airmen,  and  Coast  Guard 
personnel  operating  in  hostile  environments. 

The  current  Planning,  Programming,  Budgeting  and  Execution 
(PPBE)  process  seems  to  compound  funding  instability  even  more. 
Under  current  guidelines  for  Program  Objective  Memorandum  (POM) 
development,  Total  Obligation  Authority  (TOA)  controls  are  provided 
by  the  Office  of  the  Secretary  of  Defense  (OSD)  to  the  military  depart¬ 
ments.  As  a  POM  is  developed  by  the  military  departments,  programs 
are  prioritized  and  funded  such  that  available  TOA  at  that  control  num¬ 
ber  is  allocated  to  programs.  As  that  is  done  for  each  year  in  the  Future 
Years  Defense  Plan  (FYDP),  the  completed  POM  contains  programs 
that  consume  all  available  TOA  for  all  FYDP  years.  That  POM  is  then 
forwarded  to  OSD  forthe  joint  Cost  Assessment  and  Program  Evaluation 
/Comptroller  program  and  budget  reviews.  These  reviews  result  in  OSD¬ 
generated  Resource  Management  Decisions  (RMD)  that  promulgate 
senior  OSD -level  decisions  to  adjust  the  FYDP  input  accordingly  in  a 
zero  sum  construct.  In  other  words,  if  additional  program  funding  is 
added  to  one  program  in  an  RMD,  then  an  associated  offset  is  applied 
to  other  programs.  At  the  end  of  the  OSD/OMB  reviews,  decisions  and 


Defense  ARJ,  April  3013,  Vol.  SO  No.  1: 071-098 


88 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


associated  funding  levels  in  the  FYDP  are  rolled  into  the  President’s 
Budget  (PB).  Hence,  the  PB  FYDP  reflects  programs  that  consume  the 
total  available  TOA  for  each  FYDP  year. 

The  PB  then  traverses  the  Congressional  Enactment  Process,  result¬ 
ing  in  program  adjustments  via  congressional  marks  and  funds  being 
appropriated.  The  apportionment/allocation/allotment  process  distrib¬ 
utes  appropriated  funding  for  execution  against  these  original  program 
requirements.  So  acquisition  program  managers  would  now  have  the 
budget  authority  needed  to  execute  their  programs. 

But  wait,  was  the  appropriation  amount  the  number  needed  to 
execute  the  program?  Did  we  not  mention  program  adjustments  were 
potentially  made  at  all  levels  between  the  program  office  and  the  appro¬ 
priation  (Program  Executive  Office  [PEO],  Service  Headquarters,  OSD/ 
OMB,  two  authorization  committees  and  one  authorization  confer¬ 
ence  committee,  two  appropriation  committees  and  the  Appropriation 
Conference  Committee,  and  maybe  a  major  subordinate  command 
included  for  good  measure)?  The  result  of  all  these  adjustments  could 
be  the  addition  or  deletion  of  funds  to  programs.  So  the  resultant  fund¬ 
ing  available  for  obligation  may  be  significantly  different  than  what  was 
requested  months  before  at  the  start  of  the  budget  process.  These  fund¬ 
ing  levels  may  force  a  technical  adjustment  and/or  a  restructure.  Each 
one  of  the  individual  increases  or  decreases  to  a  program  budget  request 
resulted  in  the  program  office  responding  with  reclamas  or  appeals  and 
program  “what  if”  drills.  At  the  end  of  the  day,  decisions  are  made  at  all 
levels  to  provide  the  most  “bang  for  the  buck”  and  to  balance  the  books; 
many,  many  programs  are  impacted,  some  positively  and  some  nega¬ 
tively.  So  what  happened  to  program  stability? 

The  program  then  enters  the  execution  phase  with  its  allocated 
funding.  Concurrent  with  execution,  the  program’s  funding  requirement 
is  updated  and  the  cycle  starts  again.  Very  few  programs  execute  at  the 
funding  level  provided  in  the  appropriation.  Real  world  events,  unknown 
at  the  time  of  the  program  estimate  or  at  the  time  of  these  incremental 
decisions,  result  in  needed  funding-level  adjustments.  Even  if  there  were 
no  changes  at  the  program  office  level,  all  the  intermediate  levels  may 
have  adjustments  to  programs  for  which  this  program  becomes  a  bill 
payer.  Or  most  likely,  a  new  requirement  from  the  warfighter  has  been 
introduced,  but  there  is  no  additional  TOA  provided  by  OSD  for  these 
new  requirements.  So  the  PEO,  or  the  major  subordinate  command,  or 
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the  military  department,  or  OSD/OMB,  or  the  congressional  commit¬ 
tees  will  prioritize  existing  program  needs  along  with  new  program(s) 
funding  needs.  If  program  requirements  were  not  previously  funded  and 
now  are  of  sufficient  priority  to  require  funding,  other  programs  in  a  like 
amount  of  funding  (remember  we  had  TOA  controls  for  each  year  in  the 
previous  F  YDP  that  consumed  the  available  funding)  must  move  into  the 
unfunded  category.  And  that  something  might  be  a  number  of  programs 
that  are  decremented  to  obtain  the  required  funds  for  the  new  initiatives. 
Additionally,  there  could  be  a  number  of  new  warfighting  requirements 
and/or  technology  enhancements  introduced  into  this  budget  cycle, 
which  would  likewise  be  prioritized  and  potentially  result  in  many  more 
programs  becoming  full  or  partial  bill  payers. 

So  what  happened  to  funding  stability?  In  a  process  that  allows  and 
encourages  technology  changes,  requirements  changes,  and  funding 
changes,  how  can  there  ever  be  stable  programs  and  stable  funding?  The 
author’s  hypothesis  is  that  funding  stability  is  simply  not  probable  given 
the  above  arguments. 

So  let’s  again  turn  to  the  DAU  survey  results  on  what  financial  man¬ 
agement  practitioners  believe  would  enable  funding  stability  (Table  2). 


Note.  GO  =  General  Officer 
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Two  Top  Enablers— Good  Planning  &  Program  Execution, 
Clear  Requirements 

We  might  expect  to  see  stable  requirements,  better  management 
of  new  technology,  and  a  more  efficient  way  to  fund  new  requirements 
without  impacting  legacy  or  existing  programs  as  key  enablers  to  fund¬ 
ing  stability.  However,  the  data  suggest  at  the  practitioner  level  that  good 
planning  and  program  execution,  and  a  clear  definition  of  requirements 
would  go  a  long  way  to  stabilizing  funding.  Program  office  personnel 
might  consider  external  agency  adjustments  to  their  programs  a  direct 
result  of  poor  planning  and  program  execution.  The  plan  is  used  to 
defend  budget  requirements;  accordingly,  a  poor  plan  results  in  pro¬ 
gram  reductions.  The  same  can  be  said  for  program  execution,  as  higher 
headquarters  at  all  levels  of  oversight  are  watching  financial  execution 
to  ensure  its  synchronization  with  the  plan.  When  programs  cannot 
execute  in  accordance  with  the  plan,  analysts  at  all  levels  will  see  the 
opportunity  to  realign  funds  to  higher  priority  programs. 

Third  Top  Enabler— Good  Communications 

Good  Communications  is  directly  linked  to  the  first  two  enablers. 
As  one  develops  the  program  plan  for  execution,  constant  and  effective 
communication  with  oversight  agencies  enables  a  better  understanding 
of  the  plan  by  all  concerned.  This  allows  program  office  personnel  to 
strengthen  that  plan  by  documenting  identified  deficiencies  and  defend- 
ingthem  against  program  adjustments.  Direct  communications  between 
program  financial  personnel  and  internal  program  staff  personnel  will 
result  in  better  justification  materials  as  well  as  better  reclamas  (for 
adjustments  from  Service  and  OSD  headquarters)  and  appeals  (con¬ 
gressional  adjustments)  to  proposed  reductions.  Good  Communications 
between  program  office  personnel  and  oversight  personnel  will  facilitate 
more  defensible  justifications  as  well  as  abetter  understanding  of  pro¬ 
gram  nuances  by  all  the  stakeholders. 

Fourth  Top  Enabler— Honesty  and  Trust  the  Process 

Honesty  and  Trust  the  Process  as  an  enabler  provides  interesting 
insights  into  the  perspective  of  many  financial  management  profes¬ 
sionals.  The  financial  culture  for  many  years  has  centered  on  protecting 
funding  and  an  unwillingness  to  return  funds  to  a  higher  headquarters  for 
use  against  higher  priority  programs.  In  many  cases,  that  culture  results 
in  funds  being  wasted  or  oversight  personnel  discovering  disconnects 
between  the  plan  and  the  execution,  as  well  as  reallocating  those  funds 
and  exacerbating  stability.  Being  honest  with  higher  headquarters  and 
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freeing  up  dollars  do  result  in  funding  stability  as  this  honesty  will  enable 
unneeded  funds  to  be  made  available  for  higher  priority  needs  of  the 
Service.  Releasing  unneeded  funds  for  higher  priority  programs  enriches 
the  enterprise  by  helping  to  fix  other  program  problems,  and  creates  an 
attitude  of  “you  helped  me  with  this  program,  I  will  help  you  when  you 
need  it.”  Program  managers  need  to  take  an  enterprise  perspective  and 
offer  up  unnecessary  funds  for  high-priority  programs,  and  headquarters 
managers  need  to  support  that  culture  with  payback  as  required. 

Obviously,  high-priority  programs  and  those  with  high  levels  of 
visibility  generally  are  funded  in  accordance  with  their  priority.  These 
programs  tend  to  be  more  stable,  and  funding  issues  are  resolved  quickly 
by  senior  leaders.  Army  Digitization  was  the  number  one  priority  pro¬ 
gram  for  a  number  of  years  in  the  late  1990s  and  could  count  on  receiving 
the  funding  it  needed.  The  Mine  Resistant  Ambush  Protected  (MRAP) 
vehicle  has  enjoyed  that  priority  over  the  last  few  years.  This  enabler 
could  be  linked  with  political/personal  influence.  High-priority  pro¬ 
grams  tend  to  have  the  visibility  and  a  strong  proponent  to  promote  the 
program.  Sometimes  these  proponents  are  powers  in  the  political  arena 
and  sometimes  they  are  simply  key  personnel  in  the  decision-making 
community  that  usher  the  program  through  the  system.  A  strong  backer 
at  the  right  level  can  do  wonders  for  program  stability.  Often,  senior 
leaders  become  supporters  of  programs  and  ensure  they  are  properly 
resourced  due  to  their  position.  As  an  example,  some  years  ago  the  Vice 
Chief  of  Staff  of  the  Army  was  a  strong  soldier  system  proponent  and 
would  ensure  subordinate  levels  would  not  decrement  soldier  programs 
without  his  concurrence.  At  a  later  time,  the  same  could  be  said  for  a  Vice 
Chief  who  was  a  strong  aviation  program  proponent.  Years  before  that 
was  an  Army  Digitization  supporter.  So  obviously,  having  a  key  stake¬ 
holder  in  the  right  place  at  the  right  time  in  the  process  can  minimize 
funding  turbulence. 

Programs  with  good  cost  estimates  and  good  program  execution  con¬ 
sistent  with  those  estimates  would  be  more  stable  as  internal  shortfalls 
do  not  drive  instability.  However,  external  detractors  would  continue 
to  impact  such  programs.  The  current  focus  on  funding  to  the  OSD 
Independent  Cost  Estimate  (ICE)  should  assist  in  minimizing  cost  tur¬ 
bulence.  More  often  than  not,  the  ICE  will  result  in  a  higher  cost  estimate 
than  the  Program  Office  Estimate. 
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Working  the  system  provides  another  interesting  insight.  Program 
managers  that  understand  the  PPBE  process  as  well  as  the  Congressional 
Enactment  Process  are  more  inclined  to  be  engaged  throughout  and  make 
these  systems  and  processes  work  for  them.  As  an  example,  program  man¬ 
agers  who  know  when  key  management  decisions  are  being  made  in  these 
two  processes,  and  are  proactively  involved  in  being  available  on  short 
notice  to  provide  needed  data  or  information,  seem  to  fare  better  than 
those  who  sit  back  and  await  the  outcome  of  the  process.  Program  man¬ 
agers  who  walk  the  halls  in  the  Pentagon  identifying  the  latest  pending 
action  on  their  programs  can  be  extremely  helpful  in  providing  informa¬ 
tion  to  decision  makers  and  their  staffs.  Knowledge  of  the  Congressional 
Enactment  Process  as  to  when  committee  and  subcommittee  marks  are 
taking  place  and  responding  with  appeal  information  contributes  enor¬ 
mously  to  program  stability  by  reducing  program  adjustments. 

The  same  can  be  said  for  knowledgeable  and  experienced  people 
on  the  program  management  staff  who  can  fight  off  these  adjustments 
based  upon  their  knowledge,  understanding,  and  relationships  with  key 
stakeholders.  Their  knowledge  of  key  system  characteristics,  the  con¬ 
tracting  and  resourcing  processes,  and  their  technical  competence  can 
be  enormously  influential  in  successfully  protecting  programs. 

A  program  that  is  directly  related  to  the  war  effort  should  enjoy 
funding  priority  and  program  stability.  Many  programs  support  the  war 
effort,  but  again,  some  are  more  critical  than  others.  The  MRAP  program 
has  entertained  high  priority  as  it  saves  lives  and  minimizes  injury.  Any 
program  related  to  IED  threat  reduction  will  enjoy  a  high  priority  for 
funding.  Again,  soldier  systems  would  fit  into  this  category  and  enjoy 
minimal  funding  adjustments. 

This  author  was  quite  surprised  that  multiyear  procurements  did 
not  have  a  higher  number  of  votes  on  the  survey.  Clearly,  programs  with 
approved  multiyear  procurements  will  enjoy  stable  funding  due  to  the 
high  termination  costs  associated  with  those  programs.  Multiyear 
Procurement  is  associated  with  Economic  Order  Quantity  purchases 
where  contractors  are  encouraged  to  buy  in  quantity  or  affect  production 
efficiencies  in  exchange  for  a  contract  covering  a  number  of  years— nor¬ 
mally  3  to  5.  Decision  makers  are  reluctant  to  decrement  these  programs 
as  a  commitment  was  made  to  fund  them.  The  downside  to  multiyear 
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procurements  is  they  reduce  the  flexibility  of  the  Service,  OSD/OMB, 
and  Congress  due  to  the  large  termination  costs.  This  author  would  have 
expected  this  to  be  the  number  one  enabler  to  stable  funding. 

Programs  necessary  to  maintain  an  industrial  base  capability  would 
tend  to  be  exempt  from  program  adjustments  for  the  primary  purpose 
of  ensuring  the  nation  has  that  capacity  in  time  of  need.  The  reason  the 
survey  had  so  few  responses  for  this  factor  is  most  likely  attributable  to 
few  programs  fitting  this  situation.  A  follow-on  survey  identifying  this 
as  a  potential  enabler  should  result  in  many  more  identifying  this  factor 
as  a  significant  funding  stability  enabler. 

Program  Stability  and  Information  Technology 

A  report  to  Congress  entitled,  “A  New  Approach  for  Delivering 
Information  Technology  Capabilities  in  the  Department  of  Defense” 
offers  dramatic  approaches  to  funding  IT  that  might  provide  funding 
stability  (DoD,  2010).  The  executive  summary  states  that  anticipated 
legislative  changes  will  be  required  to  fully  implement  the  new  acquisi¬ 
tion  process.  It  goes  on  to  suggest  a  single  appropriation  of  IT  projects 
where  research  and  development,  procurement,  and  operations  and 
maintenance  will  all  be  performed  using  this  single  appropriation. 

Single  Appropriation 

This  approach  would  certainly  contribute  to  funding  stability  while 
providing  enormous  flexibility  in  managing  programs.  Obviously,  all 
programs  would  enjoy  a  greater  degree  of  funding  stability  under  such 
a  proposal.  It  will  be  interesting  to  see  if  Congress  will  support  such  a 
proposal  in  the  PB  for  IT  projects.  Unfortunately,  this  author  does  not 
see  Congress  applying  such  a  proposal  to  all  programs  as  it  significantly 
reduces  their  oversight  responsibility. 

Revolving  Fund 

The  second  IT  approach  is  to  create  a  revolving  fund  similar  to  the 
National  Defense  Sealift  Fund.  Funding  would  be  deposited  in  a  nonex¬ 
piring  account  with  obligation  authority  for  the  purposes  under  the  act. 
Again,  this  approach  would  provide  funding  stability  for  IT  programs 
and  has  applicability  to  other  programs  as  well.  It  will  be  interesting  to 
see  how  Congress  reacts  to  this  approach  if  the  Department  of  Defense 
tries  to  implement  it  in  a  future  PB. 
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Stable  Funding  Through  IT  Funding  Elements 

Stable  Funding  through  IT  Funding  Elements  is  the  third  approach, 
which  would  use  a  single  funding  element  such  as  a  program  element  (PE) 
or  procurement  line  item  to  fund  a  portfolio  of  similar  projects.  “Funding 
for  the  combination  of  smaller  IT  projects  may  be  best  addressed  by  a 
stable  budget  defined  by  a  single  funding  element”  (DoD,  2010,  p.  7).  This 
funding  element  would  support  an  IT  capability  in  lieu  of  individual  pro¬ 
grams.  Again,  this  will  require  congressional  approval  as  the  proposal 
reduces  congressional  approval  and  oversight  for  defense  programs.  The 
Army  Digitization  Office  in  the  mid  to  late  1990s  had  such  a  funding  line 
where  Congress  appropriated  $100  million  of  Research,  Development, 
Test  and  Evaluation  funds  for  competing  digitization  programs.  That 
program  was  examined  in  detail  by  numerous  oversight  agencies  and  was 
abandoned  due  to  dissatisfaction  with  the  rigor,  which  was  extremely 
detailed,  and  the  lack  of  congressional  oversight. 

These  three  approaches  to  IT  programs  could  prove  very  effective 
in  giving  the  department  the  necessary  flexibility  to  meet  the  unique 
requirement  related  to  maturing  technologies  that  require  near  real-time 
reaction  to  fully  implement.  Those  same  flexibilities  are  necessary  to 
provide  funding  stability  to  all  programs. 
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Discussion  and  Recommendations 

It  is  this  author’s  opinion  that  funding  stability  in  the  real  world 
environment  where  changing  requirements,  technologies,  and  pro¬ 
gram  funding  are  the  norm  to  meet  warfighting  needs  is  most  likely 
a  myth.  It  cannot  occur  without  some  major  change  in  the  PPBE  and 
Congressional  Enactment  processes,  both  of  which  are  unlikely.  One 
could  envision,  albeit  not  easily,  a  military  department  reserving  in  the 
FYDP  a  portion  of  the  TOA  for  future  programs  or  for  program  adjust¬ 
ments.  Doing  that  of  course  would  expose  that  TOA  for  OSD/OMB  use 
to  be  applied  to  other  programs.  The  risk  of  losing  those  funds  would 
most  likely  not  be  supported  by  the  Service  leadership.  And  of  course, 
the  military  departments  have  many  more  requirements  for  funding 
than  available  TOA,  so  they  would  be  required  to  defund  programs  to 
reserve  this  TOA.  Highly  unlikely! 

Recent  passage  of  the  Budget  Control  Act  of  2011  exacerbates  this 
issue  by  requiring  significant  reductions  to  defense  funding  along  with 
the  potential  for  across  the  board  reductions  (sequestration)  if  Congress 
cannot  reach  agreement  on  further  deficit  reduction.  Funding  instability 
for  many  defense  programs  will  most  likely  occur— whether  there  is  an 
agreement  or  a  lack  of  agreement. 

So  the  acquisition  community  as  well  as  oversight  agencies  will 
continue  to  study,  suggest,  recommend,  and  talk  about  the  need  to  ensure 
program  and  funding  stability.  But  real  world  implementation  of  these 
acquisition  reform  ideas  will  continue  to  elude  the  acquisition  practitio¬ 
ner  as  the  requirements  generation  and  the  resource  allocation  systems 
are  simply  not  flexible  enough  to  react  to  emerging  changes  while  main¬ 
taining  program  stability. 
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Project  management  has  been  a  constant  challenge 
for  the  U.S.  Department  of  Defense  (DoD)  acquisition 
community.  While  most  DoD  projects  are  technologically 
advanced,  the  tools  and  methods  to  manage  these  proj¬ 
ects  are  the  same  as  for  simple,  repetitive  projects.  The 
authors  argue  that  traditional  approaches  fail  because 
they  only  evaluate  the  relationships  between  two  of 
the  three  elements  of  cost,  schedule,  and  performance. 
Instead,  they  have  developed  a  system  dynamics  model 
that  allows  cost,  schedule,  and  performance  to  interact 
and  influence  one  another.  This  model  is  complemen¬ 
tary  to  other  research  and  intended  to  be  usable  by  the 
practicing  project  manager.  The  results  from  model  runs 
will  provide  consequences  for  three  potential  control 
alternatives  in  DoD  project  management. 
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The  search  for  the  ideal  form  of  DoD  project  management  has  existed 
since  the  dawn  of  DoD  projects  and  is  driven  by  a  desire  to  reduce  the 
DoD  project  failure  rate.  In  fact,  many  external  observers  as  well  as 
quite  a  few  DoD  acquisition  professionals  would  probably  say  that  more 
DoD  projects  fail  than  succeed.  Many  researchers  have  explored  the 
reasons  for  general  project  failure  in  great  detail.  They  have  developed 
many  theories  that  withstand  academic  scrutiny,  but  the  range  of  even 
simply  understanding  the  nature  of  projects  varies  depending  on  project 
characteristics,  the  project  industry,  and  where  researchers  decide  to 
restrict  their  studies.  Ultimately,  no  one  solution  makes  DoD  or  any  other 
domain  of  project  management  work.  That  the  DoD  project  manage¬ 
ment  industry  has  not  had  more  research  is  somewhat  surprising  given 
the  large  number  and  size  of  DoD  projects.  However,  this  is  a  common 
problem  in  most  project  management  domains  (Love,  Edwards,  &  Irani, 
2008).  Certainly,  DoD  could  gain  more  appreciation  and  insight  from  an 
improved  understanding  of  the  way  it  conducts  project  management, 
which  is  different  from  other  industries. 

Background 

According  to  the  U.S.  Government  Accountability  Office  (GAO),  half 
of  the  major  defense  acquisition  programs  are  not  meeting  cost  goals,  and 
80  percent  have  increasing  unit  costs  (GAO,  2011).  GAO  further  notes 
that  between  2008  and  2010,  the  98  major  defense  acquisition  projects 
have  grown  in  budget  by  9  percent  (GAO,  2011).  In  the  Fiscal  Year  (FY) 
2012  budget  request,  the  U.S.  Department  of  Defense  (Comptroller)  asked 
for  $85.3  billion  of  its  $553.1  billion  budget,  or  approximately  15.4  percent 
for  major  defense  acquisition  projects  (Comptroller,  2011). 

Frequently,  budget  pressure,  schedule  pressure,  or  changing  user 
demands  are  cited  as  the  reasons  for  both  commercial  and  DoD  failures 
(Meier,  2010).  However,  these  challenges  have  been  present  for  as  long 
as  projects  have  been  undertaken,  and  the  trend  within  DoD  is  getting 
worse  instead  of  better  (GAO,  2011).  The  fact  that  many  of  these  projects 
are  developmental  and  have  little  or  no  basis  for  comparison  is  a  fair 
excuse  for  why  initial  cost,  schedule,  and  performance  estimates  prove 
to  be  incorrect.  However,  DoD  project  management  needs  to  develop 
approaches  to  overcome  the  current  trends.  The  first  step  to  improve¬ 
ment  is  a  solid  understanding  of  how  most  DoD  project  management 
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works.  The  goal  of  the  authors’  research  is  to  ultimately  provide  insight 
into  the  practical  application  of  alternative  decisions  within  DoD  project 
management  from  the  perspective  of  the  government  project  manager. 

Project  management  has  existed  since  man  began  building  things. 
However,  many  researchers  define  the  beginnings  of  the  formal  disci¬ 
pline  of  project  management  with  the  U.S.  Polaris  missile  development 
and  use  of  critical  path  methods  in  the  1950s  (Lyneis,  Cooper,  &  Els, 
2001;  Pich,  Loch,  &  De  Meyer,  2002;  Tishler,  Dvir,  Shenhar,  &  Lipovetsky, 
1996;  Williams,  2005).  Therefore,  a  close  connection  exists  between  DoD 
projects  and  formal  project  management,  but  not  all  project  management 
is  the  same.  Tishler  et  al.  (1996)  note  that  defense  projects  are  different 
from  commercial  projects  due  to  a  larger,  more  interdisciplinary  design 
and  higher  technological  risks.  Despite  this  early  bond  between  DoD 
and  project  management,  many  U.S.  senior  DoD  leaders  as  well  as  the 
U.S.  Congress  have  expressed  the  view  that  U.S.  DoD  project  manage¬ 
ment  needed  improvement  as  early  as  1970,  and  have  since  changed  the 
acquisition  policy  guidance  nine  times  (Ferrara,  1996). 

Sorenson  (2009)  provides  an  in-depth  overview  of  the  history  of  U.S. 
DoD  acquisition.  He  notes  that  the  “current  defense  acquisition  process 
is  constructed  on  a  foundation  of  distrust”  (Sorenson,  2009).  By  this  he 
means  that  the  distribution  of  power  as  well  as  the  extensive  oversight 
is  all  in  place  to  ensure  that  everyone  is  involved  in  doing  the  right  thing, 
and  to  avoid  the  illegal  and  immoral  past  history  of  highly  publicized 
procurement  irregularities  related  to  defense  acquisition.  He  notes  that 
there  are  variations  in  how  projects  are  executed  and  decisions  that  are 
made  on  varying  projects.  Sorenson  (2009)  further  comments  that  the 
Secretary  of  Defense  has  (though  infrequently)  terminated  acquisition 
programs,  but  Congress  never  has.  He  highlights  many  of  the  problems 
with  defense  acquisition,  from  poor  cost  estimates  to  development  delays 
to  changing  requirements  to  excessive  oversight  (Sorenson,  2009). 

Other  researchers  have  found  that  many  DoD  project  managers 
underestimate  cost  and  schedule  due  to  the  failure  to  understand 
complexities  involved  as  well  as  seemingly  futile  efforts  to  correct  an 
underperforming  project,  which  often  results  in  blaming  exogenous 
variables  as  opposed  to  endogenous  ones  (Lyneis  &  Ford,  2007).  They 
further  add  that  a  great  deal  of  research  exists  noting  general  theories 
on  the  need  to  reduce  elements  of  project  management  rework  cycles, 
but  domain-specific  advice  or  research  is  limited  (Lyneis  &  Ford,  2007). 
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Ford  and  Dillard  (2009)  address  this  deficiency  using  a  system  dynamics 
model  of  the  JAVELIN  missile  development  program  that  allowed  them 
to  evaluate  the  strengths  and  weaknesses  of  “evolutionary  acquisition” 
by  comparing  two  strategies  for  system  development. 

System  dynamics  also  has  a  long  history  with  DoD.  One  of  the  earliest 
uses  of  system  dynamics  within  DoD  was  in  the  diagnosis  and  legal  sup  - 
port  for  delay  and  disruption  claims  by  Ingalls  Shipbuilding  against  the 
U.S.  Navy  in  the  development  ofthe  amphibious  assault  ships  in  the  1970s 
(Cooper,  1980).  This  model  was  effective,  but  was  taken  from  the  perspec¬ 
tive  of  the  contractor  in  the  1970s.  An  extensive  system  dynamics  model 
was  used  to  evaluate  the  general  field  of  software  development  (Abdel- 
Hamid  &  Madnick,  1991).  Black  and  Repenning  (2001)  developed  a  generic 
system  dynamics  model  based  on  a  commercial  manufacturing  new 
product  development  to  evaluate  how  early  failure  to  apply  appropriate 
resources  to  a  project  (or  multiple  projects)  results  in  a  “firefighting”  phe¬ 
nomenon  that  results  in  poor  project  performance.  Taylor  and  Ford  (2006) 
further  reinforce  this  research  with  the  same  phenomenon  and  additional 
“tipping  point”  analysis  as  applied  to  construction  management.  These 
research  results  are  highly  valuable,  but  focused  on  the  commercial  world 
that  does  differ  from  government  project  management  in  that  government 
project  management  is  more  focused  on  managing  a  contractor  who  is 
doing  the  development.  More  recent  uses  of  system  dynamics  models 
have  been  in  the  actual  prosecution  of  combat  operations  in  the  areas  of 
command  and  control,  search  and  rescue,  and  irregular  warfare  (Coyle, 
Exelby,  &  Holt,  1999;  DoD  Announces,  2008).  All  of  these  models  are  valu¬ 
able  and  insightful,  but  they  do  not  provide  practicing  project  managers 
much  specific  detail  in  ways  to  perform  their  jobs  better. 

In  the  last  3  years,  Ford  and  Dillard’s  JAVELIN  system  dynamics 
model  is  one  of  the  most  recent  models  and  does  provide  good  insight  into 
varying  acquisition  approaches  to  a  project.  Both  the  current  authors  and 
they  agree  that  a  DoD  project  manager  must  still  accomplish  a  single¬ 
block  development  even  within  the  larger  evolutionary  acquisition;  and 
attempting  to  document  and  model  all  external  influences  on  DoD  proj¬ 
ect  management  may  be  futile.  Therefore,  our  approach  is  to  develop  a 
historically  based,  empirical  model  that  produces  the  final  cumulative 
cost,  schedule,  and  performance  results  in  a  manner  that  allows  us  to 
evaluate  the  consequences  of  three  simple  control  alternatives  within 
any  larger  acquisition  framework.  Thus,  our  model  could  one  day  be 
incorporated  into  Ford  and  Dillard’s  to  provide  additional  understanding 
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of  the  dynamics  of  DoD  project  management.  We  believe  our  research 
will  help  practicing  DoD  project  managers  better  understand  positive 
and  negative  consequences  of  simple  project  control  alternatives  that 
they  may  consider. 


Dynamic  Hypothesis 

If  you  were  to  ask  many  DoD  professionals  to  describe  how  acquisi¬ 
tion  truly  works  at  the  strategic  level,  they  would  describe  a  framework 
similar  to  that  depicted  in  Figure  1.  In  this  figure,  one  strategic  activity 
is  all  or  the  vast  majority  of  what  influences  the  next  downstream  strate¬ 
gic  activity.  Therefore,  a  given  threat  (or  change  in  threat)  causes  a  new 
delivered  requirement  that  changes  a  budget  estimate,  which  causes 
the  schedule  estimate,  and  the  end  result  is  a  performance  expectation. 
At  the  budget  event,  Congress  may  intervene  and  adjust  up  or  down 
the  budget,  which  impacts  the  downstream  activities.  Eventually,  the 
expected  performance  will  have  an  impact  on  the  threat.  While  this 
figure  is  relatively  simple  to  understand,  the  problem  is  that  it  ignores 
the  speed  of  change  in  many  of  these  subactivities  (i.e.,  budget  develop¬ 
ment  every  2  years  while  the  requirements  may  change  every  year  or 
less)  and  ignores  other  impacts  of  the  subactivity  interaction.  It  further 
assumes  that  a  single  change  can  be  controlled  or  managed  with  “simple 
processes”  such  as  through  the  monitoring  of  a  work  breakdown  struc¬ 
ture  or  earned  value  management.  Previous  research  has  shown  these 
techniques  may  be  effective  if  there  are  relatively  few  or  no  unknowns, 
but  they  prove  inadequate  when  there  are  many  unknowns  or  the  true 
state  of  the  variable  may  not  be  known  for  some  time  (Dvir  &  Lechler, 
2004;  Thomas  &  Mengel,  2008). 

FIGURE  1.  DOD  ACQUISITION  AT  THE  STRATEGIC  LEVEL 

I 

Threat  Requirement  Budget  Schedule  Performance 

t 

Congress 

As  an  alternative,  the  authors  suggest  that  DoD  professionals  adopt 
a  mental  framework  like  that  shown  in  Figure  2.  In  this  figure,  all  of  the 
same  activities  as  Figure  1  are  present.  However,  Figure  2  illustrates 
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that  delays  are  present  and  every  activity  impacts  every  other  activity. 
This  means  that  an  adjustment  to  the  budget  will  impact  schedule  and 
performance  over  time.  That  this  is  a  more  accurate  depiction  of  the 
real  world  is  usually  not  in  question,  but  the  major  issue  is  how  we  deal 
with  this.  The  authors  believe  that  a  system  dynamics  model  will  serve 
as  an  effective  tool  to  better  understand  what  is  going  on  and  to  propose 
an  alternative  for  improved  system  response.  In  other  words,  we  hope  to 
find  a  better  way  to  perform  DoD  proj  ect  management  for  the  practicing 
project  manager. 


FIGURE  2.  MENTAL  FRAMEWORK 


Model  Methodology 

It  is  important  to  first  note  that  the  authors  subscribe  to  the  belief 
that  DoD  project  management  as  a  system  is  poorly  understood. 
Therefore,  our  objective  is  to  better  understand  the  system  and  its 
responses  through  the  use  of  an  empirical  model.  Because  of  this  objec¬ 
tive,  our  model  is  relatively  simple  and  strives  to  show  several  aspects 
of  the  system  and  its  dynamic  behavior  over  time  that  may  help  improve 
the  overall  results.  The  model  is  shown  in  Figure  3. 

The  foundation  of  our  model  is  a  work  flow  process  surrounding 
system  design/understanding.  Our  work  flow  process  is  very  simple 
and  defines  system  design/understanding  project  work  of  any  kind  as  a 
percentage  in  one  of  three  states:  work  to  be  done,  work  in  progress,  and 
work  completed.  The  completion  of  all  work  would  equate  to  a  perfect 
understanding  of  the  system  being  acquired,  which  should  be  the  goal 
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of  all  projects.  While  we  make  a  simplifying  assumption  that  all  work 
is  equal  in  priority  and  execution,  we  believe  that  over  the  entire  project 
life  cycle,  this  is  the  ideal  case  and  appropriate  for  our  objective.  With  all 
other  influences  being  removed,  our  work  flow  will  complete  all  activi¬ 
ties  within  15  years,  which  is  appropriate  for  most  DoD  projects.  In  our 
baseline  model,  no  unknown  work  or  rework  is  included.  While  this  is 
certainly  not  accurate  nor  representative  of  the  real  world,  our  goal  is  to 
best  understand  the  ideal  case  before  moving  to  more  intricate  situations. 

To  allow  for  dynamic  consequences,  control  of  starting  and  complet¬ 
ing  work  is  done  through  a  comparison  of  the  projected  or  estimated 
budget,  schedule,  or  performance  to  the  actual  budget,  schedule,  or 
performance.  In  the  case  of  the  budget  comparison,  we  have  incorpo¬ 
rated  a  2-year  delay  due  to  the  DoD  Planning,  Programming,  Budgeting 


FIGURE  3.  WORK  FLOW  PROCESS  SURROUNDING  SYSTEM  DESIGN/UNDERSTANDING 
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and  Execution  process  where  a  project  budget  is  submitted,  and  about 
2  years  later  the  actual  approved  budget  is  delivered.  (We  use  the  terms 
“budget”  and  “cost”  interchangeably  in  this  model  for  simplicity,  and  are 
only  focused  on  total  system  design  and  production  costs,  not  the  actual 
total  life-cycle  costs  that  include  sustainment  and  disposal.)  When  the 
difference  between  the  estimated  budget,  schedule,  or  performance  and 
the  actual  budget,  schedule,  or  performance  is  positive  (i.e.,  the  project 
is  under  cost  and/or  ahead  of  schedule  and/or  less  capable  than  initially 
desired),  then  work  is  allowed  to  start  and  be  completed  at  an  acceler¬ 
ated  rate  corresponding  to  an  increase  in  work  execution.  However,  when 
the  difference  in  the  estimated  budget,  schedule,  or  performance  and 
the  actual  budget,  schedule,  or  performance  is  negative  (i.e.,  the  project 
is  over  cost  and/or  behind  schedule  and/or  more  capable  than  initially 
desired),  then  the  work  is  slowed  to  a  decelerated  rate  corresponding  to 
a  slowing  of  work  execution  or  delaying  of  work.  As  an  example,  if  the 
actual  budget  is  25  percent  over  the  estimated  budget,  then  the  work 
initiation  and  completion  rates  are  slowed  by  12.5  percent  due  to  the 
2-year  delay.  In  this  model,  budget  and  schedule  are  equally  important 
and  contribute  the  same  to  the  work  rates.  Therefore,  a  situation  with  a 
25  percent  over  budget  and  a  25  percent  over  schedule  would  result  in  a 
corresponding  37.5  percent  reduction  to  the  work  rates. 

Budget  and  schedule  flows  are  based  on  the  work  in  progress.  The 
budget  flow  is  the  product  of  work  in  progress  multiplied  by  a  cost  per 
work  constant  that  is  multiplied  by  the  ratio  of  the  number  of  systems  to 
desired  number  of  systems  and  the  ratio  of  current  performance  to  the 
desired  performance.  This  assumes  a  cost  reduction  is  associated  with 
fewer  quantities  of  systems  and  less  system  performance.  The  schedule 
flow  is  the  product  of  the  work  in  progress  multiplied  by  a  schedule  per 
work  constant.  Either  the  budget  or  schedule  flow  can  move  in  a  positive 
or  negative  direction  allowing  for  budget  or  schedule  reductions,  but  it 
is  important  to  note  that  because  some  amount  of  work  is  in  progress  at 
any  given  time,  these  flows  will  never  be  negative.  It  is  also  important 
to  note  that  budget  and  schedule  do  not  directly  influence  each  other  in 
this  model.  This  is  due  to  observations  that  a  budget  increase  does  not 
guarantee  a  reduction  in  schedule  nor  does  a  schedule  increase  guarantee 
a  budget  reduction  (assuming  all  other  factors  are  the  same). 

Previous  project  management  research  has  addressed  the  interac¬ 
tions  of  cost,  schedule,  and  work.  Many  earned  value  management  and 
earned  value  schedule  studies  have  evaluated  cost,  schedule,  and  work. 
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However,  little  to  no  previous  research  has  enabled  the  interaction  of 
cost,  schedule,  and  work  levels  in  addition  to  performance  levels  over 
time.  Our  research  provides  some  insight  into  this  interaction.  We  have 
incorporated  performance  in  our  model  by  evaluating  it  in  two  ways. 
First,  the  number  of  systems  is  another  flow  that  impacts  the  total  bud¬ 
get.  As  long  as  the  project  is  under  budget,  then  the  number  of  systems 
will  remain  the  same.  Once  the  project  goes  above  cost,  then  the  number 
of  systems  will  be  reduced  in  an  effort  to  reduce  cost  growth.  This  is  a 
common  behavior  observed  in  DoD  projects. 

Second,  performance  is  evaluated  through  a  percentage  of  the  initial 
desired  performance.  The  desired  performance  begins  at  100  percent 
and,  like  system  quantity,  as  long  as  the  project  is  under  budget,  will 
remain  at  its  current  level.  Also  like  system  quantity,  once  a  project  is 
over  budget,  the  performance  level  will  be  reduced  by  a  percentage  in 
an  effort  to  reduce  the  budget  and  schedule  of  the  system  development. 
Additionally,  the  amount  of  performance  degradation  could  also  be 
thought  of  as  a  quantifiable  estimate  of  program  risk,  which  may  not 
be  a  problem  or  be  tolerable  to  the  project  stakeholders.  Both  of  these 
performance  measures  can  be  generically  applied  and  are  helpful  in  our 
gaining  a  basic  understanding  of  how  this  model  operates.  While  both 
of  these  performance  measures  impact  the  budget  flow,  only  the  perfor¬ 
mance  level  compared  to  the  desired  performance  level  impacts  the  work 
flow.  This  is  due  to  the  assumption  that  complete  system  understanding 
can  be  gained  through  one  system  and  no  further  insight  is  gained  from 
additional  system  production. 

More  detailed  aspects  such  as  rebaselining  or  evaluating  which  per¬ 
formance  elements  are  reduced  have  been  excluded  so  that  the  essential 
model  behavior  can  be  observed.  Verification  and  validation  of  this  model 
was  done  through  two  means.  First,  common  system  dynamics  practices 
as  referenced  in  Barlas  (1996)  were  successfully  conducted.  Second,  the 
authors  have  used  a  case  study  to  validate  the  results  of  the  model  with 
actual  system  performance.  The  authors  have  chosen  the  U.S.  Army’s 
Future  Combat  System  as  the  case  study. 

Case  Study— The  Army’s  Future  Combat  System 

In  1999,  the  U.S.  Army  began  designing  the  Future  Combat  System 
(FCS)  as  a  means  of  preparing  itself  for  what  it  expected  to  be  the  future 
of  warfare.  The  Army  expected  to  have  the  first  unit  equipped  in  2011  and 
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the  entire  Army  equipped  by  2032.  FCS  involved  multiple  air  and  ground 
systems  that  were  networked  and  interoperable.  One  key  tenet  of  the  FCS 
effort  was  that  information  could  replace  mass,  and  a  second  tenet  was 
that  FCS  components  could  be  deployed  rapidly.  The  ultimate  combina¬ 
tion  was  a  highly  technical  and  revolutionary  system-of-systems  that 
sought  to  push  technology  and  balance  many  competing  priorities  (GAO, 
2008).  FCS  was  officially  terminated  as  a  program  in  the  summer  of 2009. 


While  many  unique  and  interesting  dynamics  surround  this  program, 
it  is  used  as  a  means  of  verifying  and  validating  our  model.  In  our  model, 
the  key  FCS  inputs  were  the  project  cost  and  schedule  estimates.  These 
inputs  were  taken  from  the  U.S.  Congressional  Budget  Office  (CBO,  2006) 
report  on  FCS.  The  model  set  2005  as  the  base  year,  which  was  based 
on  availability  of  fiscal  information,  and  extends  until  2025.  Our  model 
results  of  a  total  system  cost  in  2025  of  $161  billion  are  consistent  with  the 
Congressional  Budget  Service  (CBS)  estimates  of  $160.6  billion.  While 
the  CBS  may  have  been  in  error  in  its  assessment,  the  fact  that  our  model 
achieves  similar  results  instills  confidence  in  our  approach.  Additionally, 
our  model  estimates  that  when  the  project  was  killed  in  2009,  the  number 
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of  systems  would  be  reduced  to  14,  with  an  estimated  project  completion 
date  of 2021.  These  results  are  consistent  with  actual  results  of  that  time 
and  provide  the  final  validation  of  our  model  for  general  understanding. 

Results 

Using  this  model,  we  now  turn  to  evaluating  how  varying  responses 
impact  the  results.  We  have  focused  on  project  manager  responses,  and 
these  results  should  hold  true  as  long  as  DoD  projects  are  evaluated 
against  their  initial  (or  current)  estimates.  The  potential  strategies 
evaluated  are  to  Remove  Controls,  Ignore  Schedule,  Ignore  Budget,  and 
Improve  Estimate.  The  reduction  in  total  systems  remains  the  same  for 
all  strategies  so  further  discussion  of  it  is  excluded. 

Remove  Controls 

One  potential  strategy  to  improve  project  success  is  to  remove  all 
project  controls.  While  this  model  does  not  account  for  some  of  the 
uncertainties  and  unknowns  that  occur  within  the  life  of  the  project, 
it  does  help  us  evaluate  a  perfect-world  scenario.  In  this  perfect  world 
with  no  reduction  in  work  flow,  the  project  completes  all  work  by  2025, 
but  only  attains  67  percent  of  the  desired  capability.  The  total  budget  is 
$147  billion,  which  is  a  9  percent  cost  reduction  of  the  baseline. 

Ignore  Schedule 

Another  potential  strategy  to  improve  project  success  is  to  ignore  the 
schedule  comparison.  This  alternative  operates  on  the  principle  that  “if 
you  need  it  bad  enough,  you  will  do  anything  to  get  it.”  Upon  first  look, 
this  alternative  achieves  the  lowest  total  cost  at  $141  billion  (a  13  percent 
reduction  of  the  baseline)  with  all  work  completed  by  2014  and  a  perfor¬ 
mance  drop  of  21  percent  of  the  initial  desired  capability. 

Ignore  Budget 

Positive  results  were  achieved  by  ignoring  schedule  comparison  so 
the  authors  were  interested  in  what  would  happen  if  the  budget  com¬ 
parison  is  ignored.  In  this  case,  the  total  cost  was  $133  billion,  which  is 
an  18  percent  reduction  in  baseline  and  looks  very  attractive.  However, 
only  67  percent  of  the  initial  desired  performance  is  achieved  on  system 
completion,  which  does  not  occur  until  2032.  While  this  strategy  results 
in  the  best  cost  reduction,  the  performance  and  timeline  are  sacrificed. 
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Improve  Estimate 

Another  seemingly  simple  strategy  is  to  focus  all  efforts  on  knowing 
the  true  cost  and  schedule  up  front.  This  technique  is  in  consonance 
with  most  systems  engineering  literature  and  does  make  sense  with 
what  every  senior  DoD  acquisition  leader  advises.  While  not  the  focus 
of  this  research,  it  is  another  challenge  entirely  to  determine  how  to 
accomplish  this.  However,  this  approach  results  in  a  total  budget  of 
$210  billion  or  130  percent  of  the  baseline,  with  86  percent  of  the  desired 
performance  delivered. 


Work 

Work 

Total 

Performance 

Completed 

Completed 

Cost 

Delivered 

by  2025 

by 

Baseline 

$161B 

69  percent 

100  percent 

2021 

Remove  Controls 

$147B 

67  percent 

100  percent 

2025 

Ignore  Schedule 

$141B 

79  percent 

100  percent 

2014 

Ignore  Budget 

$133B 

67  percent 

77  percent 

2032 

Improve  Estimate 

$210B 

86  percent 

100  percent 

2022 

Looking  at  this  data  reveals  two  general  observations.  First,  in  a 
perfect  world  potential  opportunities  for  cost  reductions  abound,  but, 
second,  they  come  at  the  expense  of  performance  level.  The  largest 
variation  in  strategies  regarding  cost  is  30  percent,  which  could  trigger 
a  Nunn-McCurdy  breach  and  require  congressional  reporting.  However, 
the  trade-off  is  clearly  associated  with  the  amount  of  performance  deliv¬ 
ered,  which  at  most  varies  by  19  percent,  and  when  that  performance  is 
delivered,  which  varies  at  most  by  over  18  years.  Additional  statistical 
analysis  shows  that  no  two  factors  are  highly  correlated,  but  that  the 
most  likely  relationship  is  between  total  cost  and  performance  delivered. 

What  these  results  mean  to  a  DoD  project  manager  is  that  no  single 
strategy  is  likely  to  achieve  cost,  schedule,  and  performance  optimiza¬ 
tion.  Project  managers  need  to  evaluate  the  prioritized  objectives  of 
that  project’s  stakeholders  and  develop  their  strategies  to  meet  those 
priorities.  For  instance,  if  a  project  is  needed  quickly,  then  ignoring  how 
the  project  is  comparing  to  the  initial  schedule  maybe  the  best  solution. 
If  a  project  needs  high  performance,  most  likely  a  good  strategy  for  the 
project  manager  is  to  ensure  the  systems  engineering  and  analysis  is  per¬ 
formed  early  so  that  the  best  cost  and  schedule  estimates  can  be  made. 
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Future  Research  and  Conclusions 

While  these  results  are  interesting,  they  are  certainly  not  to  be  con¬ 
sidered  rigid  rules  of  DoD  project  management.  In  fact,  many  elements 
and  influences  have  been  excluded  in  this  model  in  an  effort  to  gain  an 
initial  understanding  of  the  total  system  behavior.  The  dynamics  of  how 
DoD  project  work  is  prioritized  and  executed,  the  dynamics  of  varying 
design  and  evaluation  methods,  and  the  dynamics  and  value  of  the  three- 
milestone  DoD  acquisition  gate  process  are  all  work-related  influences 
that  should  be  further  studied.  Another  potential  future  area  of  study 
is  the  combination  of  this  model  into  Ford  and  Dillard’s  evolutionary 
acquisition  comparison  to  see  if  even  simple  control  alternatives  affect 
the  results  of  the  research.  The  dynamics  of  varying  budget  delays  as 
well  as  the  impact  of  congressional  budget  action  should  also  be  further 
studied.  Finally,  the  dynamics  of  system  quantity  changes  to  the  actual 
system  cost  is  an  area  that  can  also  be  expanded  to  provide  better  fidel¬ 
ity  in  this  model. 

All  of  these  future  areas  of  study  require  significant  investigation 
and  study,  and  likely  vary  from  project  to  project.  This  further  supports 
the  authors’  theory  that  DoD  project  management  is  a  highly  contextual 
process  that  requires  dynamic  understanding  of  influences  that  some¬ 
times  do  not  make  themselves  known  for  some  time.  A  system  dynamics 
model  such  as  the  one  discussed  in  this  article  could  be  used  to  best  iden¬ 
tify  and  predict  total  project  behavior  so  that  varying  strategies  could  be 
evaluated  for  the  best  one  in  a  given  situation.  The  model  could  certainly 
be  expanded  and  complemented,  and  the  DoD  would  do  well  to  invest 
more  resources  in  exploring  why  projects  fail  (or  succeed),  documenting 
the  circumstances  and  influences,  and  distributing  them  for  widespread 
use  in  the  project  management  arena.  However,  a  single  or  even  several 
causal  factors  should  be  avoided  to  explain  all  projects  as  every  project 
is  unique  and  must  be  evaluated  in  its  unique  context. 
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Review: 

This  book  has  a  foreword  by  then  First  Sea  Lord  Admiral  Jonathan 
Band,  which  nicely  summarizes  it:  “this  excellent  and  very  readable 
account  of  the  life  [of  the  ship]...  and  the  worlds  from  which  it  sprang  and 
into  which  she  emerged.”  The  book  also  describes  the  politics  that  led  to  the 
current  build  program  of  much  larger  Carrier  Vessel  Future  (CVF)  vessels. 
Initially,  the  sorry  story  is  told  of  the  cancellation  of  the  CVA-01,  lead  ship 
of  the  first  post-war  carrier  class.  Louis  Rydill,  as  the  design  manager,  is 
quoted  as  feeling  he  was  on  the  rack  trying  to  achieve  the  capability  of  a 
Forrestal  design  within  a  displacement  of  53,000  tons  and  a  necessary, 
but  daunting  level  of  innovation.  Meanwhile,  there  was  ineptitude  by  the 
Royal  Navy’s  hierarchy  in  the  corridors  of  power  (contrasted  later  with 
the  “A  Team”  16  years  on,  which  won  the  Falklands  Campaign).  Following 
the  demise  of  British  naval  aviation,  there  was  the  subsequent  slow  and 
painful  climb  back  to  a  fleet  led  by  three  “Through-Deck  Cruisers”— only 
“carriers”  in  a  limited  sense  once  the  short  take  off  and  vertical  landing 
(STOVL)  provision  was  incorporated. 

Next,  Invincible  and  her  design  intent  is  described  (familiar  to 
those  involved,  including  this  reviewer,  who  contributed  to  Nick  Childs’ 
research,  as  well  as  coauthored  the  definitive  technical  paper  in  1980 
on  the  design— more  appropriate  for  those  who  would  like  something 
technically  detailed  rather  than  this  essentially  strategic  and  personali- 
ties-focused  history).  Childs  has  caught  the  essentially  innovative  nature 
of  this  quite  new  concept— based  on  Rydill’s  early  helicopter  cruiser  stud¬ 
ies  and  then  Tony  Austin’s  coherent  through-deck  and  all  gas  turbine 
propelled  design,  developed  without  any  previous  ship  on  which  to  base  it 
(Honnor  &  Andrews,  1982).  The  book  then  covers  the  Harrier  ramp  story 
well,  if  un-technically— there  was  a  lot  more  to  it  than  being  “just  welded 
on  to  the  forward  end  of  the  runway.” 

The  nadir  was  reached  with  the  early  Thatcher  government’s  defense 
cutting  regime  with  a  chapter  entitled  “For  Sale,”  yet  followed— merci¬ 
fully,  for  the  Navy— with  the  Falklands  redemption  of  maritime  capability 
and  a  chapter  on  “Invincible  at  War.”  This  should  be  the  key  chapter  of 
the  book,  but  is  a  mere  10  pages  covering  the  immediate  perspective  of 
the  Commanding  Officer  (then)  Captain  Jeremy  Black,  and  little  on  how 
well  the  ship  and  its  embarked  aircraft  performed.  The  subsequent  chap¬ 
ter  “What  Lessons”  focuses  on  the  strategic  rather  than  the  tactical-  or 
design-related  lessons. 

The  last  part  of  the  book  addresses  post-Cold  War  peacekeeping,  where 
the  ability  of  carriers  to  project  power  ashore  was  shown  to  be  limited 
with  the  Invincibles  and  thus  made  the  case  for  the  two  big  (65,000-ton) 
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carriers.  On  the  vexing  question  of  naval  ship  costs,  despite  the  fact  that 
it  dominates  the  whole  fleet  acquisition  process,  Childs  does  not  ask  why 
the  Queen  Elizabeth  Class  (QEC)  carriers,  which  are  large  but  slow  and 
minimally  armed,  have  been  subject  to  ever  rising  costs.  Clearly,  each 
major  program  has  been,  successively,  the  only  significant  United  Kingdom 
shipbuilding  program,  and  therefore  had  to  shoulder  the  added  burden  of 
sustaining  that  national  capability.  However,  this  writer  believes  the  cost 
escalation  is  also  due  to  the  persistent  but  false  belief  by  politicians  and 
wider  government  (including  the  naval  hierarchy)  that  “value-for-money” 
could  be  delivered  by  giving  industry  the  responsibility  for  designing  and 
project  managing  naval  shipbuilding,  instead  of  leaving  both  in-house. 

However,  none  of  the  above  is  addressed  in  Childs’  book,  which  largely 
focuses  on  the  contributions  of  a  succession  of  senior  naval  officers.  It  is 
a  fascinating,  if  dispiriting,  story  of  sacrificing  the  size  of  the  rest  of  the 
fleet  to  keep  carrier  aviation  alive.  The  uphill  nature  of  this  struggle  seems 
consistent  with  the  sense  of  a  nation  that  has  collectively  lost  sight  of  its 
maritime  raison  d’etre.  However,  this  is  only  part  of  the  story  that  Childs 
could  have  addressed,  as  his  story  has  a  worrying  lack  of  engagement  with 
the  underpinning  engineering  narrative.  Could  that  be  coincidental? 
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rial  from  primary  sources,  including  program  documents,  policy  papers,  memoranda, 
surveys,  interviews,  etc.  Articles  are  characterized  by  a  systematic  inquiry  into  a  subject 
to  discover/revise  facts  or  theories  with  the  possibility  of  influencing  the  development  of 
acquisition  policy  and/or  process. 

We  encourage  prospective  writers  to  coauthor,  adding  depth  to  manuscripts.  It  is  rec¬ 
ommended  that  a  mentor  be  selected  who  has  been  previously  published  or  has  expertise  in 
the  manuscript’s  subject.  Authors  should  be  familiar  with  the  style  and  format  of  previous 
Defense  ARJ s  and  adhere  to  the  use  of  endnotes  versus  footnotes,  formatting  of  reference 
lists,  and  the  use  of  designated  style  guides.  It  is  also  the  responsibility  of  the  corresponding 
author  to  furnish  a  government  agency/employer  clearance  with  each  submission. 


MANUSCRIPTS 


Manuscripts  should  reflect  research  of  empirically  supported  experience  in  one  or 
more  of  the  areas  of  acquisition  discussed  above. 

Research  articles  maybe  published  both  in  print  and  on-line,  or  as  a  Web-only  ver¬ 
sion.  Articles  that  are  4,500  words  or  less  (excluding,  abstracts,  biographies,  and  endnotes) 
will  be  considered  for  both  print  as  well  as  Web  publication.  Articles  between  4,500  and 
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10,000  words  will  be  considered  for  Web-only  publication,  with  an  abstract  included  in  the 
print  version  of  the  Defense  A  RJ.  In  no  case  should  article  submissions  exceed  10,000  words. 

Audience  and  Writing  Style 

The  readers  of  the  Defense  ARJ  are  primarily  practitioners  within  the  defense  acqui¬ 
sition  community.  Authors  should  therefore  strive  to  demonstrate,  clearly  and  concisely, 
how  their  work  affects  this  community.  At  the  same  time,  do  not  take  an  overly  scholarly 
approach  in  either  content  or  language. 

Format 

Please  submityourmanuscriptwithreferences  in  APA format  (author-date-page  num¬ 
ber  form  of  citation)  as  outlined  in  the  Publication  Manual  of  the  American  Psychological 
Association  (6th  Edition).  For  all  other  style  questions,  please  refer  to  the  Chicago  Manual 
of  Style  (15th  Edition). 

Contributors  are  encouraged  to  seek  the  advice  of  a  reference  librarian  in  completing 
citation  of  government  documents  because  standard  formulas  of  citations  may  provide 
incomplete  information  in  reference  to  government  works.  Helpful  guidance  is  also  avail¬ 
able  in  Garner,  D.  L.,  and  Smith,  D.  H„  1993,  The  Complete  Guide  to  Citing  Government 
Documents:  A  Manual  for  Writers  and  Librarians  (Rev.  Ed.),  Bethesda,  MD:  Congressional 
Information  Service. 

Pages  should  be  double-spaced  and  organized  in  the  following  order:  title  page, 
abstract  (120  words  or  less),  two-line  summary,  list  of  key  words  (five  words  or  less),  body 
of  the  paper,  reference  list  (works  cited),  author’s  note  (if  any),  and  any  figures  or  tables. 
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Figures  or  tables  should  not  be  inserted  (or  embedded,  etc.)  into  the  text,  but  segre¬ 
gated  (one  to  a  page)  following  the  text.  When  material  is  submitted  electronically,  each 
figure  or  table  should  be  saved  to  a  separate,  exportable  file  (i.e.,  a  readable  EPS  file).  For  addi¬ 
tional  information  on  the  preparation  of  figures  or  tables,  see  CBE  Scientific  Illustration 
Committee,  1988,  Illustrating  Science:  Standards  for  Publication,  Bethesda,  MD:  Council 
of  Biology  Editors.  Restructure  briefing  charts  and  slides  to  look  similar  to  those  in  previ¬ 
ous  issues  of  the  Defense  ARJ. 

The  author  (or  corresponding  author  in  cases  of  multiple  authors)  should  attach  to 
the  manuscript  a  signed  cover  letter  that  provides  all  of  the  authors’  names,  mailing  and 
e-mail  addresses,  as  well  as  telephone  and  fax  numbers.  The  letter  should  verify  that  the 
submission  is  an  original  product  of  the  author;  that  it  has  not  been  previously  published  in 
another  journal  (monographs  and  conference  proceedings,  however,  are  okay);  and  that  it 
is  not  under  consideration  by  another  journal  for  publication.  Details  about  the  manuscript 
should  also  be  included  in  this  letter:  for  example,  title,  word  length,  a  description  of  the 
computer  application  programs,  and  file  names  used  on  enclosed  CDs,  e-mail  attachments, 
or  other  electronic  media. 


COPYRIGHT 


The  Defense  ARJ  is  a  publication  of  the  United  States  Government  and  as  such  is 
not  copyrighted.  Because  the  Defense  ARJ  is  posted  as  a  complete  document  on  the  DAU 
homepage,  we  will  not  accept  copyrighted  manuscripts  that  require  special  posting 
requirements  or  restrictions.  If  we  do  publish  your  copyrighted  article,  we  will  print  only 
the  usual  caveats.  The  work  of  federal  employees  undertaken  as  part  of  their  official  duties 
is  not  subject  to  copyright  except  in  rare  cases. 

Web-only  publications  will  be  held  to  the  same  high  standards  and  scrutiny  as  articles 
that  appear  in  the  printed  version  of  the  journal  and  will  be  posted  to  the  DAU  Web  site 
at  www.dau.mil. 

In  citing  the  work  of  others,  please  be  precise  when  following  the  author-date-page 
number  format.  It  is  the  contributor’s  responsibility  to  obtain  permission  from  a  copyright 
holder  if  the  proposed  use  exceeds  the  fair  use  provisions  ofthelaw(see  U.S.  Government 
Printing  Office,  1994,  Circular  92:  Copyright  Law  of  the  United  States  of  America,  p.  15, 
Washington,  D.C.).  Contributors  will  be  required  to  submit  a  copy  of  the  writer’s  permis¬ 
sion  to  the  Managing  Editor  before  publication. 
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Policy 

We  reserve  the  right  to  decline  any  article  that  fails  to  meet  the  following  copy¬ 
right  requirements: 

•  The  author  cannot  obtain  permission  to  use  previously  copyrighted  material 
(e.g.,  graphs  or  illustrations)  in  the  article. 

•  The  author  will  not  allow  DAU  to  post  the  article  in  our  Defense  ARJ issue  on  our 
Internet  homepage. 

•  The  author  requires  that  usual  copyright  notices  be  posted  with  the  article. 

•  To  publish  the  article  requires  copyright  payment  by  the  DAU  Press. 


SUBMISSION 


All  manuscript  submissions  should  include  the  following: 

•  Cover  letter 

•  Biographical  sketch  for  each  author  (70  words  or  less) 

•  Headshot  for  each  author  should  be  saved  to  aCDR  disk  as  a  300  dpi  (dots  per  inch) 

or  high-print  quality  JPEG  or  Tiff  file  saved  as  no  less  than  5x7.  Please  note:  images 
from  Web,  PowerPoint,  or  e-mail  will  not  be  accepted  due  to  low  image  quality. 

•  One  copy  of  the  typed  manuscript,  including: 

°  Title  (12  words  or  less) 

°  Abstract  of  article  (120  words  or  less) 

°  Two-line  summary 
°  Keywords  (5  words  or  less) 

°  Document  excluding  abstract  and  references  (4,500  words  or  less  for  the  printed 
edition  and  10,000  words  or  less  for  online-only  content) 

These  items  should  be  sent  electronically,  as  appropriately  labeled  files,  to  Defense 
ARJManaging Editor,  Norene  Fagan-Blanch  at:  Norene.Fagan-Blanch@dau.mil. 
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